Escribir un comentario

This review is the result of a virtual, collaborative Live Review discussion organized and hosted by PREreview on June 25, 2026. The call was joined by 22 people: 3 facilitators from the PREreview Team (Dr. Daniela Saderi, Sol Ruiz, Pía Tavella), 1 Expert Moderator (Dr. Christopher Steven Marcum), 8 discussion participants (including Dr. Raffaele Vardavas, who participated in the discussion but did not contribute to compiling this report), and 9 listeners. The authors of this review dedicated additional asynchronous time over two weeks to compose this final report using notes from the Live Review. We thank all participants who contributed to the discussion and made it possible for us to provide feedback on this preprint.

Summary

This retrospective cohort study evaluates the safety of three 2023-2024 COVID-19 vaccine formulae (Pfizer-BioNTech, Moderna, and Novavax) among 7,560,772 United States Medicare beneficiaries aged 65 years and older. The study examines the association between these vaccines and 15 pre-specified adverse events, spanning cardiovascular, thrombotic, neurological, and immunological outcomes, using administrative claims data from September 11, 2023, to April 6, 2024, based on a subset of the Medicare Fee-for-Service (FFS) dataset, with controls from the Biologics Effectiveness and Safety (BEST) Initiative, a prior United States Food & Drug Administration (FDA) research protocol.

The central question being explored is whether these vaccines are safe for public health when used routinely in clinics. The authors used a self-controlled case series (SCCS) design and a conditional Poisson regression model to estimate the incidence rate ratios (IRRs) and attributable risks (ARs), buttressed by additional statistical adjustments (i.e., event-dependent observation time and positive predictive value (PPV)-based multiple imputation).

Across all independent analyses, no statistically significant increase in risk was observed for any of the pre-specified medical conditions after appropriate, unbiased statistical adjustments. While a statistically significant increase in anaphylaxis was observed among patients who received the Pfizer-BioNTech vaccine, this signal disappeared after accounting for outcome misclassification and other covariates.

The authors acknowledged several limitations. For example, the study used claims-based outcomes, which can often result in outcome miscalculation. Positive Predictive Value-based interpretation can facilitate the identification of mistakes in categories, but it does not fully and accurately account for those classification errors, especially when dealing with a restricted dataset. Therefore, undetected risk and bidirectional biases may be present in the findings. While adverse effects were minimal, in a high-risk elderly Medicare population no less, these findings should not be interpreted as a claim that vaccinations are foolproof. Rather, there was no detectable increase in population-level incidence following vaccination.

Nevertheless, the thorough considerations and precautions taken in this protocol and the extensive explanations of the decisions that were made in conducting the analysis in this paper, especially regarding the data limitations, are an outstanding example of how to execute and report such a large-scale study. Ultimately, this article offers a near-gold-standard benchmark that reviewers can confidently use to evaluate other vaccine safety manuscripts. This approach strengthens the impact of the ultimate conclusion of the paper: “FDA continues to believe that the benefits of COVID-19 vaccination outweigh the risks.”

Feedback and list of major and minor concerns:

There are no major concerns regarding this manuscript resulting from the Live Review. The study is professionally designed, well handled, and well written. The presented data support the claims made by the authors, and the limitations and technical decisions were transparently communicated. The dataset analysis is robust, and the conclusions are presented accurately without overgeneralizing beyond the data.

However, several minor concerns could be addressed to further improve the study. These include:

Background

• Explain in greater detail why these indications were chosen for exploration by incorporating the justification detailed in the previous paper or the study protocol. Also, clarify from the literature any pathophysiologic connection to the three vaccines examined (mRNA-based vs. protein subunit).

• In the Data Sources, Study Population, and Study Period section, it was mentioned that “Medicare FFS (Medicare Parts A and B, but not Part C)” but for international readers this categorization (A, B, and C) is not clear. Explain that Medicare Fee‑for‑Service (FFS), also called Original Medicare, covers:

i) Part A (Hospital Insurance)

ii) Part B (Medical Insurance)

iii) Part C (Medicare Advantage): Not part of FFS, offered by private insurance companies approved by CMS (Centers for Medicare & Medicaid Services)

Data & Methods

• Provide further evidence as to why a two-tailed hypothesis test, self-controlled case series (SCCS) analyses, and a Poisson regression were used. While these are robust methods, and do provide a valid model for the large dataset, other alternatives could be used (e.g., Cox regression, marginal structural models, Propensity score weighting, inverse probability weighting). Incorporating complementary analyses or detailing why these specific methods were selected would strengthen the robustness of the paper.

• A further analysis of interest would be to look at those who were vaccinated for COVID-19, yet do not otherwise accept vaccination to evaluate whether unique contributions to this specific family of vaccines had differential adverse effects.

• The specific clinical and administrative codes (ICD-10-CM, CPT, HCPCS, NDCs) are referenced as being available in the accompanying study protocol (Reference 25), which is necessary for exact replication. The authors should include these as an appendix rather than incorporating them by reference.

Results

• The Charlson Comorbidity Index is mentioned in the descriptive results section and appears in supplementary eTable 3, but it is not properly defined or identified within the main text, the methods section, or the supplementary table legend.

• The authors should address whether the confidence intervals (CI) obscure a weak but present signal associated with myocarditis as other studies, including a study from the same authors, have reported myocarditis as an adverse reaction to earlier COVID-19 vaccines in younger individuals? It would be helpful if the authors added a paragraph in the results and conclusions with a few citations or references and some explanatory text reconciling the results of the paper in older adults with the literature, and perhaps discuss potential limitations of the methods, if appropriate. (Pillay, J., Gaudet, L., Wingert, A., Bialy, L., Mackie, A. S., Paterson, D. I., & Hartling, L. (2022). Incidence, risk factors, natural history, and hypothesised mechanisms of myocarditis and pericarditis following COVID-19 vaccination: living evidence syntheses and review. BMJ : British Medical Journal (Online), 378 https://doi.org/10.1136/bmj-2021-069445; Oster ME, Shay DK, Su JR, et al. Myocarditis Cases Reported After mRNA-Based COVID-19 Vaccination in the US From December 2020 to August 2021. JAMA. 2022;327(4):331–340. doi:10.1001/jama.2021.24110; Husby A, Hansen JV, Fosbøl E, Thiesson EM, Madsen M, Thomsen RW, Sørensen HT, Andersen M, Wohlfahrt J, Gislason G, Torp-Pedersen C, Køber L, Hviid A. SARS-CoV-2 vaccination and myocarditis or myopericarditis: population based cohort study. BMJ. 2021 Dec 16;375:e068665. doi: 10.1136/bmj-2021-068665. PMID: 34916207; PMCID: PMC8683843; Altman NL, Berning AA, Mann SC, Quaife RA, Gill EA, Auerbach SR, Campbell TB, Bristow MR. Vaccination-Associated Myocarditis and Myocardial Injury. Circ Res. 2023 May 12;132(10):1338-1357. doi: 10.1161/CIRCRESAHA.122.321881. Epub 2023 May 11. PMID: 37167355; PMCID: PMC10171307).

Figures/Tables

• To aid readability and remove ambiguity, add a color-coded key, with the colors assigned to the different indications being explored.

• The supplementary eFigure 1 was not mentioned in the main text of the manuscript.

• The figures and tables are quite descriptive. It would also be helpful to provide more detailed annotations and captions to all figures. In addition, clearer references to figures and tables should be made in the text for better representation of the data/findings.

• Figure 1 is appropriate for a concept but could be improved for visual appeal. The authors should clarify what “washout” means in a more detailed caption.

Code and Data Availability

• There is a data availability statement: “All data produced in the present study are available upon reasonable request to the authors.” However, the authors should clarify whether they used the publicly accessible FFS dataset from CMS: https://www.cms.gov/medicare/payment/medicare-advantage-rates-statistics/ffs-data-2015-2024 or if they used a more restricted dataset.

• The authors indicate that: “All analyses were conducted using R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria) and SAS v. 9.4 (SAS Institute Inc., Cary, NC, United States). “However, all analytic code should be made available with the paper.

Citations & References

• The manuscript repeatedly cites Reference 25 for methodological details, which is the study protocol itself. The authors should cite the original primary literature for these methods rather than relying solely on the protocol document.

• The imputation reference to Rubin and Schenker (1986) is outdated and should be supplemented with a more recent citation.

• References 8 and 38 are identical duplicates, while References 17 and 34 are also identical, with Reference 17 remaining uncited in the text (these should be reconciled and deduplicated).

• Please add URLS and DOIS (where available) to all references, especially: 13, 23, 25, 36, and 47

• A canonical citation to the Charlson Comorbidity Index should be provided. We suggest: Charlson ME, Pompei P, Ales KL, MacKenzie CR (1987). "A new method of classifying prognostic comorbidity in longitudinal studies: development and validation". Journal of Chronic Diseases. 40 (5): 373–383. doi:10.1016/0021-9681(87)90171-8. PMID 3558716.

Concluding remarks

We thank the authors of the preprint for posting their work openly for feedback. We also thank all participants on the Live Review call for their time and for engaging in the lively discussion that led to this review. We encourage the FDA and all federal scientists to continue to post preprints of their research to facilitate timely, relevant, and transparent feedback on their work.

Competing interests

Christopher Steven Marcum is the Chair of PREreview Advisory Committee and oversaw scientific integrity for the Biden-Harris Administration.

Use of Artificial Intelligence (AI)

The authors declare that they did not use generative AI to come up with new ideas for their review.