The reliability of preprints has been a persistent source of debate in scientific publishing. Skeptics argue that findings posted without peer review are inherently untrustworthy; advocates counter that the preprint-to-publication pipeline, while imperfect, rarely produces wholesale reversals.
A large-scale analysis posted on bioRxiv by Hao Yin and Ruslan Rust (University of Southern California and Western University) provides the most comprehensive quantitative answer yet. Using Claude Sonnet 4.6 to classify changes in primary claims across 72,644 pairs of bioRxiv preprints (2018–2025) and their corresponding published journal articles, the study finds that the vast majority of preprint conclusions survive peer review substantially intact.
The numbers
Of the 72,644 preprint-journal pairs analyzed:
- 39.9% had unchanged primary conclusions
- 50.0% had minor revisions, rewording, qualification, or slight adjustment of the central claim
- 10.2% had major revisions, claims substantively altered, effects reversed, or conclusions shifted from positive to null
The trend has been improving: major revisions declined from 17.0% in 2019 to 5.7% in 2024. The decline could reflect changing preprint culture: early COVID-era posts were rushed, while today many authors incorporate peer review-style revisions before their first preprint posting.
Hedging language also shifted. In 85.6% of pairs, the certainty level of the primary claim stayed the same. Where it changed, authors were roughly twice as likely to become more cautious (8.4%) than more confident (4.2%).
Retraction rates
Preprint-associated papers had significantly lower retraction rates than papers that never appeared as preprints. Among papers first posted on bioRxiv, the retraction rate was 8.1 per 10,000 papers. Among papers never posted as preprints, the rate was 18.7 per 10,000, a ratio of 2.31 (95% CI 1.20–4.45, p = 0.003).
The authors caution that this is an observational finding, not proof that preprinting causes lower retraction risk. Papers posted as preprints may differ from non-preprinted papers in ways that also affect quality; for example, authors who choose to preprint may be more confident in their results.
Field and time variation
Major revision rates varied substantially by field. Bioinformatics had the lowest rate at 7.2%, while microbiology had the highest at 17.5%. Review time also correlated with changes: papers that took the longest to publish (roughly 416 days median) had a 14.1% major revision rate, compared with 7.0% for the fastest-reviewed papers (roughly 110 days).
When the primary claim changed, 90% of secondary claims also changed, suggesting that a core finding revision tends to cascade through the paper rather than being an isolated correction.
Limitations
The study has important caveats. It analyzed abstracts only; changes in methods, figures, or full-text results were not captured. It compared the first preprint version with the published version, so it cannot separate author-driven revisions from peer-review edits. The analysis used a single LLM, which introduces its own classification errors, though validation showed agreement with domain experts (Cohen’s kappa 0.63–0.66, matching inter-expert agreement). Preprints that never reached publication were excluded, which may bias the sample toward more credible papers. And the study covers bioRxiv only; results may not generalize to medRxiv or other servers.
The Nature news article covering the study was written by Mohana Basu.
Sources
[1] Yin, H. & Rust, R. “Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models.” bioRxiv (2026). DOI: 10.64898/2026.06.30.735556
[2] Basu, M. “Think preprints are unreliable? Analysis of 70,000 studies might change your mind.” Nature (2026). https://www.nature.com/articles/d41586-026-02167-3