The manufacturers of Covid-19 vaccines say they are 95% effective. Peter Doshi re-examined the evidence and estimates they are only 19-29% effective. This pre-print of an as-yet unpublished re-analysis raises many questions but doesn’t support the claims being made on antivaccine sites.
A correspondent asked for my opinion about a study that he said has been appearing widely on COVID-19 vaccine disinformation sites with claims that it proves that the vaccines are not safe or that it shows there is “a 300% greater chance of landing in the hospital from the vaccine than from Covid itself”. I read the study. In the first place, I don’t think it showed any such thing. In the second place, there were several things about the study that made me distrust it:
- It had not yet been peer-reviewed
- It cited data from VAERS as if it were a trustworthy, meaningful source, which it isn’t; adverse events are listed on VAERS with no evidence that they were caused by the vaccine
- It failed to mention the huge body of other evidence that overwhelmingly supports the efficacy and safety of the vaccines after extensive real-world experience (over 200 million doses administered in the US alone); an unbiased review of the scientific evidence must look at the entire body of published evidence rather than choosing one or a few studies that support one’s beliefs
- Statistics can be manipulated and misinterpreted; if you don’t like the results you got, you can torture the data and pretty much make it confess to anything
AESI and the Brighton Collaboration
The study is titled “Serious adverse events of special interest following mRNA vaccination in randomized trials”. I questioned why some adverse events would be “of special interest”. The term “special interest” comes from a priority list developed by the Brighton Collaboration and endorsed by the World Health Organization (WHO).
As they explain, the Safety Platform for Emergency vACcines (SPEAC) Project generated a priority list of potential adverse events of special interest (AESI) for safety monitoring based on one or more of the following criteria:
- Known association with immunization or a specific vaccine platform
- Occurrence during wild-type disease as a result of viral replication and/or immunopathogenesis
- Theoretical association based on animal models
Wait, what? A “theoretical association based on animal models”? Is that a reasonable criterion or mere speculation and opinion? What constitutes “special interest”?
I found the study difficult to read and very confusing. It presented no new data. The authors simply reviewed the studies that the vaccine manufacturers had submitted to the FDA when they applied for emergency use authorization (EUA). They tried to identify the adverse effects reported in those studies that were listed as adverse effects of special interest (AESI) in the report of the Brighton Collaboration. If I understand correctly, they did a re-count. The manufacturers had counted the number of individuals who had had adverse effects, but an individual could have more than one AESI so the authors of this study counted each AESI as a separate event.
By their revised accounting:
Pfizer and Moderna mRNA COVID-19 vaccines were associated with an increased risk of serious adverse events of special interest, with an absolute risk increase of 10.1 and 15.1 per 10,000 vaccinated over placebo baselines of 17.6 and 42.2 (95% CI -0.4 to 20.6 and -3.6 to 33.8), respectively. Combined, the mRNA vaccines were associated with an absolute risk increase of serious adverse events of special interest of 12.5 per 10,000 (95% CI 2.1 to 22.9). The excess risk of serious adverse events of special interest surpassed the risk reduction for COVID-19 hospitalization relative to the placebo group in both Pfizer and Moderna trials (2.3 and 6.4 per 10,000 participants, respectively).
Apparently the Covid vaccine disinformation folks interpreted this to mean the vaccines are unsafe and you are 300% more likely to be hospitalized if you are vaccinated. In my opinion, that represents poor reading comprehension and motivated reasoning.
Peter Doshi
One of the authors, Peter Doshi, is a senior editor at the British Medical Journal (BMJ) and an associate professor of pharmaceutical health services research at the University of Maryland School of Pharmacy. He has long been calling for independence and transparency in COVID-19 vaccine-related decision making, for instance here. He questions the trustworthiness and meaningfulness of the reported efficacy results of the mRNA vaccines.
I agree with Doshi that independence and transparency are essential in scientific research, and that the raw data should be released. He has raised several valid concerns about the manufacturers’ methodologies that cry out for more investigation. I am willing to accept that the reported 95% efficacy results are wrong, but I suspect that Doshi’s estimates of 19% and 29% efficacy are also wrong, maybe even wronger.
This is serious stuff. A pandemic is a public health emergency. Without effective treatment, people will die. The public must have access to accurate information. Releasing raw data from the trials would be a step in the right direction. Doshi is right about that.
Conclusion: The waters are muddier
I’m afraid this new study confuses the issue rather than clarifying it. Someone is engaging in shenanigans with statistics. Is it the vaccine manufacturers? Is it the authors of this new study? Neither? Both? I leave it to you to decide. I’m not a statistician. I hope people more knowledgeable about statistics will chime in with their informed opinions.
This article was originally published in the Science-Based Medicine Blog.
