As a communications scholar, I am particularly interested in public communications about scientific matters. Even though we all had science classes in school, we were usually taught that science delivers the “correct answers” if you merely use the right methods. In reality, “doing science” is not straightforward; rather, it is a messy process, full of fits and starts, dead-ends, noisy data, and genuine controversy. Controversy in science is a part of the process, not an indication that science is malfunctioning; doing science takes time.
Misconceptions about the scientific process often lead to significant misunderstandings and complications when scientific findings become the focus of attention in the spheres of politics and policy. This disconnect between the communication of scientists and the rest of society is illustrated in the controversy concerning the use of hydroxychloroquine to treat COVID-19 patients.
Trump and hydroxychloroquine
President Trump triggered the current controversy when he opined at a press conference that the drugs hydroxychloroquine and azithromycin when taken together could well be a “game-changer” in the treatment of COVID19. This initial claim was further articulated in the following March 21 tweet.
HYDROXYCHLOROQUINE & AZITHROMYCIN, taken together, have a real chance to be one of the biggest game changers in the history of medicine. The FDA has moved mountains – Thank You! Hopefully they will BOTH (H works better with A, International Journal of Antimicrobial Agents)…..
— Donald J. Trump (@realDonaldTrump) March 21, 2020
This enthusiastic endorsement of hydroxychloroquine, a drug long used for treating malaria, rheumatoid arthritis, and lupus, was stimulated by the publication of a small exploratory French study. The study only had 21 patients in which the group treated with hydroxychloroquine recovered more quickly than patients in the control group. However, Dr. Anthony Fauci, the director of the National Institute of Allergy and Infectious diseases, said that while the study was suggestive, much more research was necessary.
In the coming weeks, the controversy surrounding hydroxychloroquine escalated, especially after a Veterans Administration sponsored study found that patients receiving hydroxychloroquine had higher mortality rates than the patients who only received supportive care. President Trump then switched to his “what have people got to lose” argument. He also announced that the government had stockpiled some 29 million doses of hydroxychloroquine. Dr. Fauci continued to caution in an interview that medical research “still needs to do the definitive studies to determine whether any intervention, not just this one, is truly safe and effective.”
Types of “definitive” studies
So, what kind of definitive studies was Dr. Fauci referring to? Some think that one only needs to give people the medicine and then determine if patients recover. People frequently share anecdotes of how their grandmother’s chicken soup cures the common cold. However, the question arises, “compared to what?” The vast majority of people recover from a given incident of disease. Even with COVID19, the recovery rate has been calculated to be as high as 98%. How does one distinguish between ordinary recovery and an expedited recovery?
Adding a control group or comparison group that receives a placebo or alternate treatment is the standard experimental feature for controlling for time-related factors that can affect illness progression and outcomes. Naturally-occurring changes related to time will be captured in the control group and thus provide a baseline for comparison. A control group also enables investigators to capture any incidental events that take place during the study and may influence health outcomes of interest.
The previously mentioned Veterans Administration study had a comparison group. However, the groups were constructed after the fact using medical records: patients who received hydroxychloroquine to treat COVID-19 were compared with patients who had received only supportive care. Retrospective studies, however, have a significant limitation: we can’t be certain that the compared groups are equivalent in terms of their underlying health (i.e., comorbidities related to COVID-19), lifestyle habits, or how sick they were upon admission to the hospital. Differences in any of these background factors could explain the observed outcome differences between comparison groups.
Some medical researchers measure known background differences between retrospectively constructed groups that are known to affect health outcomes. They then use statistical controls (i.e., covariate analysis) to account for these group differences. An observational study of this sort was conducted at a New York Medical Center and was published on May 7th in the New England Journal of Medicine. This study included 1,376 confirmed cases of COVID-19, and controlled for at least 14 factors known to influence COVID19 outcomes and mortality. Those factors included “age, race and ethnic group, body-mass index, diabetes, underlying kidney disease, chronic lung disease, hypertension, baseline vital signs, Pao2:Fio2, and inflammatory markers of the severity of illness.” The authors found that the hydroxychloroquine group did no better or worse than the comparison group when the comorbidities were controlled for.
This observational study was a significant improvement over the Veterans Administration study in accounting for health status and comorbidities, but statistical controls can’t account for unknown background factors that might influence the speed of recovery and likely mortality. Likewise, the statistical adjustment can only be approximate and easily skewed by imperfect measurement of background variables. To correct for known and unknown differences between compared groups, experimentalists prefer to randomly assign subjects to one of the experimental treatment groups before the study begins. Random assignment has been shown to create roughly equivalent groups on both known and unknown background factors if one has adequate group sizes (i.e. more than 30 subjects in the compared groups).
Studies that combine a comparison group and random assignment are called Randomized Control Trials (RCT). RCT studies are regarded as the definitive tool for determining whether medical treatments are effective and safe. On June 5, news from a large RCT trial on the efficacy of hydroxychloroquine were released in Britain. This news of the study compared 1,542 people randomly assigned to the hydroxychloroquine treatment group with 3,132 patients randomly assigned to a no treatment group. Patient outcomes were compared over a 28-day period. Compared to the control group, the hydroxychloroquine group did not have lower death rates or shorter hospital stays. Dr. Marvin Landry, one of the principal investigators of the study, minced no words regarding the meaning of the findings: “This (i.e., hydroxychloroquine) is not a treatment for COVID-19. It doesn’t work. This result should change medical practice worldwide. We can now stop using a drug that is useless.”
However, even the British study does not answer a related question of whether hydroxychloroquine can prevent the development of COVID19 in people if the drug is given prior to people becoming symptomatic. Even here hydroxychloroquine earns a grade of no confidence: a June 3 publication of a University of Minnesota-sponsored RCT study reports that people who received hydroxychloroquine after being exposed to COVID19 were not less likely to develop active infections than similarly exposed people in a control group. In short, there is no evidence that hydroxychloroquine is a “game changer” in the treatment of COVID19.
Lessons learned
What can we learn from this case? The big take-away for me is that humility and patience are virtues. Answering questions about an unknown virus during an international pandemic is a fraught process. Approving the emergency use of hydroxychloroquine as a treatment for COVID19 by the FDA was sensible and prudent given the limited information that we had, especially since the drug is very inexpensive and widely available. However, it makes no sense to treat the results of exploratory research as anything other than “suggestive.” President Trump desperately grasped at a thread of possibility and preempted the national protocols for testing the efficacy of drug treatments and vaccines for infectious diseases.
We also learn that it takes some time to obtain reliable answers on these such questions. Scientific research is a powerful tool for reducing uncertainty, but it takes time to obtain even provisional answers. It is foolhardy to take tempting shortcuts, rush the process, or to make sweeping policy decisions on tentative data. Patience is a virtue; indeed, it is really the only choice. Rushing to a premature judgment is a recipe for creating confusion, chaos, and public harm.
–30–