BACKGROUND: Quality of the peer-review process has been tested only in small studies. We describe and summarize the randomized trials that investigated interventions aimed at improving peer-review process of biomedical manuscripts. METHODS AND RESULTS: All randomized trials comparing different peer-review interventions at author-, reviewer-, and/or editor-level were included. Differences between traditional and intervention-modified peer-review processes were pooled as stand-ardized mean difference (SMD) in quality based on the definitions used in the individual studies. Main outcomes assessed were quality and duration of the peer-review process. Five-hundred and seventy-five studies were retrieved, eventually yielding 24 randomized trials. Eight studies evaluated the effect of interventions at author-level, 16 at reviewer-level, and 3 at editor-level. Three studies investigated interventions at multiple levels. The effects of the interventions were reported as mean change in review quality, duration of the peer-review process, acceptance/rejection rate, manuscript quality, and number of errors detected in 13, 11, 5, 4, and 3 studies, respectively. At network meta-analysis, reviewer-level interventions were associated with a significant improvement in review quality (SMD, 0.20 [0.06 to 0.33]), at the cost of increased duration of the review process (SMD, 0.15 [0.01 to 0.29]), except for reviewer blinding. Author-and editor-level interventions did not significantly impact peer-review quality and duration (respectively, SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for quality, and SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for duration). CONCLUSIONS: Modifications of the traditional peer-review process at reviewer-level are associated with improved quality, at the price of longer duration. Further studies are needed. REGISTRATION: URL: https://www.crd.york.ac.uk/prospero; Unique identifier: CRD42020187910.

Effects of experimental interventions to improve the biomedical peer-review process. A systematic review and meta-analysis / Gaudino, M.; Bryce Robinson, N.; Di Franco, A.; Hameed, I.; Naik, A.; Demetres, M.; Girardi, L. N.; Frati, G.; Fremes, S. E.; Biondi-Zoccai, G.. - In: JOURNAL OF THE AMERICAN HEART ASSOCIATION. CARDIOVASCULAR AND CEREBROVASCULAR DISEASE. - ISSN 2047-9980. - 10:15(2021). [10.1161/JAHA.120.019903]

Effects of experimental interventions to improve the biomedical peer-review process. A systematic review and meta-analysis

Frati G.;Biondi-Zoccai G.
2021

Abstract

BACKGROUND: Quality of the peer-review process has been tested only in small studies. We describe and summarize the randomized trials that investigated interventions aimed at improving peer-review process of biomedical manuscripts. METHODS AND RESULTS: All randomized trials comparing different peer-review interventions at author-, reviewer-, and/or editor-level were included. Differences between traditional and intervention-modified peer-review processes were pooled as stand-ardized mean difference (SMD) in quality based on the definitions used in the individual studies. Main outcomes assessed were quality and duration of the peer-review process. Five-hundred and seventy-five studies were retrieved, eventually yielding 24 randomized trials. Eight studies evaluated the effect of interventions at author-level, 16 at reviewer-level, and 3 at editor-level. Three studies investigated interventions at multiple levels. The effects of the interventions were reported as mean change in review quality, duration of the peer-review process, acceptance/rejection rate, manuscript quality, and number of errors detected in 13, 11, 5, 4, and 3 studies, respectively. At network meta-analysis, reviewer-level interventions were associated with a significant improvement in review quality (SMD, 0.20 [0.06 to 0.33]), at the cost of increased duration of the review process (SMD, 0.15 [0.01 to 0.29]), except for reviewer blinding. Author-and editor-level interventions did not significantly impact peer-review quality and duration (respectively, SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for quality, and SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for duration). CONCLUSIONS: Modifications of the traditional peer-review process at reviewer-level are associated with improved quality, at the price of longer duration. Further studies are needed. REGISTRATION: URL: https://www.crd.york.ac.uk/prospero; Unique identifier: CRD42020187910.
File allegati a questo prodotto
File Dimensione Formato  
Gaudino_Effects-experimental-interventions_2021.pdf

accesso aperto

Note: https://www.ahajournals.org/doi/suppl/10.1161/JAHA.120.019903
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 1.25 MB
Formato Adobe PDF
1.25 MB Adobe PDF Visualizza/Apri PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1578534
Citazioni
  • ???jsp.display-item.citation.pmc??? 3
  • Scopus 5
  • ???jsp.display-item.citation.isi??? 5
social impact