A basic explanation of the study results and clinical trial statistics
Outcome definitions:
- A “primary outcome” of a clinical trial is the outcome that is used to help decide how many participants need to enroll in that clinical trial
- “Secondary outcomes” are also important outcomes, but they are often less common, which is why they are not used as the primary outcome.
Our objective was to understand if any of the medications prevented severe COVID-19. When we started the study, it seemed appropriate to use oxygen levels on home oxygen monitors as one definition of severe COVID-19 in our binary, 4-part composite primary outcome.
However, after the trial started, oxygen levels no longer appeared to be a good indication of severe COVID-19 because of accuracy issues with the oxygen monitors and taking the measurements. However because we had already included this in our primary outcome, we couldn’t change the primary outcome. The primary outcome is the only outcome we could test for being "statistically significant," or "definitive."
Statistical significance: Because randomized clinical trials are considered the most rigorous ways to assess clinical trials, statistically significant outcomes are generally considered to be definitive. Thus, statistical analysis must be done with care.
A statistical test called a “p value” is the typical way of deciding if a research finding is statistically significant. A p value that is < 0.05 is used to indicate that a research finding is not just due to random chance.
If researchers analyze clinical trial data in 20 ways, or 20 outcomes, one of them would be statistically significant just by random chance. This could make an intervention appear effective based on just one analysis, and for that one finding to be "definitive."
More information about these ideas are provided here:
The 3 medications were tested separately against others who were enrolled at the same time (so when the same variant of the virus was circulating), but who were not assigned to that medication. The absolute event rates for a given medication cannot be compared to another because persons were enrolled during different variant waves of the pandemic (and different rates of vaccination).
To summarize the findings for each medication:
- Metformin did not prevent the primary outcome (reported low oxygen values, emergency room visit, hospitalization, or death).
- We did observe a 42% reduction in the key secondary outcome (emergency room visits, hospitalizations, or death).
- Because this was a secondary outcome, we can’t say it is “statistically significant” or “definitive”
- Ivermectin did not prevent the primary outcome
- We observed no reduction in the key secondary outcome of fewer ER visits, hospitalizations, or death.
- Fluvoxamine did not prevent the primary outcome
- We observed no reduction in key the secondary outcome of fewer ER visits, hospitalizations, or death.
None of the medicines changed how quickly the COVID-19 symptoms resolved over time.
Explaining some of the aspects of the factorial design
2 by 3 factorial trials like this are like running 3 separate trials simultaneously. Often the medications tested in factorial studies are published in separate manuscripts.
Due to the time-sensitive nature of the pandemic, we wanted to publish them all at once.
Here we further explain a few of the figures that are in the manuscript.
Accessing the Data
Use this link to the R file with code to replicate the analyses in the main text of the primary paper. View the Redcap survey link for the dataset request.
Next Steps
The COVID-OUT Team is working to help other trials that are assessing early treatments for COVID-19. Currently that includes the ACTIV-6 trial.