Measurement Notes: Monitoring Covid-19 Signals
The Covid pandemic continues to demand attention and action. Two months ago, I started a series of notes on measurement, sparked by regular weekly calls and presentations with colleagues here in Wisconsin. This blog extracts a few of the key points, for reference.
The Wisconsin Department of Health Services (DHS) offers summary tables, plots and maps to the public on its Covid web pages, https://www.dhs.wisconsin.gov/covid-19/data.htm. DHS also gives the public access to many of the numbers it is using. DHS is doing a good job providing daily and weekly updates to the public, while continuing to develop technical tools to help state and county leaders make decisions and act responsibly.
One key measure tracked by the DHS is the per cent of tests that are positive for Covid. “The World Health Organization recommends a positive test rate of less than 10%. The countries most successful in containing COVID have rates of 3% or less.” (https://covidactnow.org/us/wi/county/dane_county)
The display shows data from the DHS website. The bottom panel shows the per cent positive daily tests since the beginning of the pandemic in Wisconsin. The shaded area indicates the last 14 days of the series. The 4% horizontal reference line is the median value for the 30 days prior to the shaded points. In early June, the display shows values below the 3% cited by the World Health Organization. It is easy to see that recent values are above 4% and that a shift upward occurred after the middle of June.
Covid data series have missing values, delays in reporting, data dumps that represent revisions, and inconsistent definitions across reporting entities and across time. Simple data tools like the run charts shown in the panel plot tend to be less upset by data quality issues than more complex tools.
Many Covid series show a strong day-of-week signals; usually, week-end days have lower counts than Monday through Friday values. You can see the weekly wave in the display; cycles of length seven are easy to see in the total daily plot panel.
The Wisconsin DHS uses seven-day moving windows to address the day-of-week signals. Here’s a view of the same test data, in seven-day buckets.
System Notes
Monitoring for Covid-19 infection has been a challenge for large and small systems throughout Wisconsin. Here’s a definition of a system: “A system is an interdependent group of items, people, or processes with a common purpose….In a system, not only the parts but the relationships among the parts become opportunities for improvement.” (G. Langley et al. The Improvement Guide, 2012, p. 37). County governments, college campuses, city departments, school districts, businesses, and religious organizations are all systems.
The definition of system is general. A sub-system is a component of a larger system, contributing to the purpose of the larger system. A system typically has a control sub-system that enables navigation in space and time—are we making progress in achieving the purpose, at what rate? Measurement is part of a control sub-system; a control sub-system also has a component to translate signals into action/interventions.
A system design must appeal to the people who are part of the system and match their skills. This holds for measurement systems, too! One version of the ‘People Principle’ states the situation succinctly: “No matter how well a system or solution is conceived, designed and executed, if people don’t like it, it will fail. No matter how poorly a system or solution is conceived, designed or executed, if people want it to work, it will succeed.” (attributed to Al Shapiro, Ohio State University, in Breakthrough Thinking Workshop notes, 1994).
Political Reality—Context for Measurement and Monitoring
There is no unanimity about the proper balance of actions to assure public health and economic health in the United States or in Wisconsin or in any county in the state.
Before June, Covid infections, illness and death were concentrated either in urban areas that appear blue on political maps or in red rural areas with marginalized populations: meat-packing plants staffed by recent immigrants, prisons, and nursing homes. Local system leaders and influencers may have seen Covid as ‘not a big problem for us’ and any effective measurement system design must acknowledge this perspective. While Covid infections and consequences have spread more widely in the last two months in the U.S., politics continue to influence data interpretation and messaging.
American libertarian tradition and current levels of distrust in government leadership make it unlikely that testing and tracing approaches used in countries like South Korea and Germany will work the same way in the U.S. The op-ed by Keith Humphreys in Washington Post on May 14 outlined the case. Random tests to estimate parameters in epidemiological models cannot deliver good information with high refusals to participate.
Data Display
I built the Covid test displays with a Shiny app that explores Wisconsin Covid data, https://iecodesign.shinyapps.io/COVID_WI_state/. R code available on GitHub.
