A recent study posted to the medRxiv* preprint server investigated the impact and trustworthiness of trial preprints on coronavirus disease 2019 (COVID-19)-related decision-making.
Study: The trustworthiness and impact of trial preprints for COVID-19 decision-making: A methodological study. Image Credit: fizkes/Shutterstock
This news article was a review of a preliminary scientific report that had not undergone peer-review at the time of publication. Since its initial publication, the scientific report has now been peer reviewed and accepted for publication in a Scientific Journal. Links to the preliminary and peer-reviewed reports are available in the Sources section at the bottom of this article. View Sources
The COVID-19 pandemic saw the adoption of preprint servers by the scientific community, which allowed researchers to disseminate their study findings before they were published in peer-reviewed journals. However, the skepticism that preprints may contain crucial errors that may be absent in published studies, has made the scientific community more cautious regarding the adoption of such preprints.
About the study
The researchers of the present study assessed the difference between the characteristics of preprints and published trial reports. They also evaluated key sections of the studies, namely the methods and results and their impacts on COVID-19 policy and decision making.
The team used the search strategy of systematic reviews and network meta-analyses (SRNMAs) that included conducting daily searches in the COVID-19 database of the World Health Organization (WHO). This database is a comprehensive collection of published reports and preprint literature related to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), sourced from across the globe.
Articles eligible for the study were preprints and peer-reviewed studies which reported trials that included randomized patients having suspected, possible, or confirmed COVID-19 to receive either drug treatments, antiviral antibodies, cellular therapies, a placebo, or standard clinical care. Trials that randomized healthy cohorts that were either exposed or unexposed to SARS-CoV-2 to prophylactic drugs, placebo, or standard clinical care were also included in the present study.
Pairs of trained and calibrated reviewers worked independently using a pilot-tested form. They obtained data related to differences in key methods and results of preprints and published trial studies. The team of reviewers also compared the features and risk of bias of trials associated with preprints, trials initially posted as a preprint and subsequently published, and trials that were published.
The team evaluated the median time required for a preprint report to be published in a journal. They also assessed the source of funding, the number of centers and participants involved, the intensity of care, severity of disease studied, the statistical significance of primary or secondary outcomes, and the risk of bias.
Results
The study results showed that among the 356 eligible trials, 101 were preprints, 250 were journal publications, and 74 were initially available as preprints and were subsequently published.
In comparison to trials published without preprints, preprints and trials that were available as preprints before being published had a higher chance of being registered. On the other hand, trials that were only available as preprints had higher odds of reporting interim results, describing drug therapies than either antiviral antibodies, cellular therapies, or prophylaxis, and receiving industry funding. Furthermore, trials that were first posted as preprints before being subsequently published were more likely to receive government funding.
One-fourth and almost half of the total preprints eligible for the study were published in peer-reviewed journals three and six months post the initial availability of the preprint, respectively. A year after the initial availability, one-third of the total preprints remained unpublished. Moreover, preprints that were funded by the government, studied inpatients, or reported on severe patients, were published faster than the ones that did not receive any government funding, studied outpatients, or included patients suffering from mild or moderate disease, respectively.
Approximately 56.8% of the total preprints were found to have one or more discrepancies in their key methods or results as compared to their subsequent published trial report. The most common discrepancy reported in the key methods was the characterization of allocation concealment and in the key results were differences in the outcome data between publications and preprints.
The study also reported other differences in key methods including the reporting of one or more statistics that were not reported in the preprint, the reporting of the interim results in the preprint, and the inclusion of a protocol and/or statistical analysis plan in the published report.
Conclusion
The study findings showed that there was no convincing evidence stating that preprint trial reports provide results that are less reliable or less trustworthy than published trial reports. Moreover, the study results showed that preprints could affect meta-analyses results and encourage users to consider data from preprints for rapid decision-making in contexts that are subject to peer reviews.
This news article was a review of a preliminary scientific report that had not undergone peer-review at the time of publication. Since its initial publication, the scientific report has now been peer reviewed and accepted for publication in a Scientific Journal. Links to the preliminary and peer-reviewed reports are available in the Sources section at the bottom of this article. View Sources
Article Revisions
- May 12 2023 - The preprint preliminary research paper that this article was based upon was accepted for publication in a peer-reviewed Scientific Journal. This article was edited accordingly to include a link to the final peer-reviewed paper, now shown in the sources section.
Genetic risk factors for long-COVID uncovered in a large multi-ethnic study