As a stats geek, I am bugged by some of the reporting of the numbers around the coronavirus pandemic. So, I decided to write a post explaining what you, the non-stats-geek, should pay attention to when it comes to coronavirus reporting.
Raw numbers need comparisons.
If I were to use time-and-space travel to put you into a closed room in a location and month unknown to you, and then tell you “The temperature outside is 32 degrees,” it would be essentially meaningless to you. Is that higher or lower than yesterday? What about the day or week before? And so on.
By the same token, learning that a given state had a certain number of COVID-19 deaths in a day is, by itself, not very informative. It’s only when you add more numbers in a series that you begin to get real information. (This is the move from data to information, which is an important concept but beyond the scope of this post. Look up “data information knowledge” for more info.)
So, Kentucky having 11 deaths reported on Monday is certainly bad news, but whether it means “getting worse” or “getting better” is not possible without something to compare it to.
Raw numbers need context.
The next step is to go from a simple trend line to putting the numbers in context. Using our opening example, if I told you the daily temperatures had been 45 – 40 – 38 – 32, you could say “it’s getting colder” with certainty. But to gain meaning beyond the trend, you would need the greater context. In this case, the context would be comparing those temperatures to the normal temperatures for that location and month. Those temperatures during the winter would not be too shocking. Those temperatures in July would mean something unusual or even historic is happening.
If Smithville has 10 COVID deaths in a day, and Jonesville has 30, that means Jonesville is sicker overall than Smithville, right? Not necessarily. Without context, all you can say is that Jonesville had a higher amount of deaths than Smithville.
But if you add the context of population, for example, you have more valid basis on which to draw conclusions. If Smithville has a population of 200 and 10 deaths, that’s a mortality rate of 5%. Meanwhile, Jonesville has a population of 1,000 and 30 deaths, that’s a mortality rate of 3%. So, Jonesville is actually sicker than Smithville.
Raw numbers need the right context.
So far, what I’ve covered is fairly non-controversial. But now, we need to get down to cases. Test cases and COVID cases, that is.
Most states are reporting raw COVID case numbers, as in “we have 1,000 cases of COVID-19.” It’s shorthand for “confirmed cases,” but we need to be clear and not use the shorthand – because there are, no doubt, many more actual cases out there.
The problem, of course, is the number of actual tests being run, and the conditions under which they are run. One county in Colorado is going to test everyone in the county. Guess what? Their ratio of confirmed cases to tests is going to be much more accurate than, say, Kentucky’s.
And, if you only run tests on people that are showing symptoms, or that know they’ve been exposed, then you will get a higher ratio of confirmed cases to tests administered – BUT, you will have a lower overall infection rate, because you are testing less of the overall population.
So, talking about infection rates is pretty meaningless if your comparisons are based on different test protocols and different percentages of the population being tested.
Governor Beshear’s graph about flu infection rates is a good graph, with good meaning. But comparing our raw number of cases to other states is not helpful without the context of overall population and overall tests performed.
What I would like to see
If I could design the reporting of COVID stats, I would like to see results of the following:
- Percentage of the state’s population tested
- Percentage of those tests coming back positive
- Percentage of the state’s population tested and positive
In addition, projections of cases and deaths need to be both raw numbers and percentages, so we have some context.
In another story on the site, we have cross-posted the story by Al Cross of KY Health News about the various models and what they project. Granted, all models at this point are more or less good guesses, since we have so little test data as a nation. But, the presentation of those models is more valid and more helpful, in my opinion, because of the use of overall population, both of people and of beds.
–30–