Communications of the ACM
Online Game Aims to Improve Scientific Peer Review Accuracy
Credit: University of Texas at Austin
Peer review of scientific research is an essential component of research publication, the awarding of grants, and academic promotion. Reviewers are often anonymous. However, a new study by researchers at the Johns Hopkins Bloomberg School of Public Health found that greater cooperation between reviewer and author can improve accuracy of the review. Their study is published in the Nov. 9 edition of the journal PLoS ONE.
To examine the accuracy of different review processes, the Johns Hopkins researchers developed a model using an online game on the Amazon E2cloud. Participants were asked to solve and review questions from the GRE (Graduate Record Examinations). The study examined both closed review, in which the author did not know the reviewers, and open review, where the author knows the reviewers.
The study found that when review behavior was public and under open review, cooperative interactions increased 13 percent. Overall accuracy between closed and open review models was similar. However, reviewers and authors who participated in cooperative interactions had an 11 percent higher reviewing accuracy rate.
"Our results suggest that increasing cooperation in the peer review process could reduce the risk of reviewing errors," says Jeffrey Leek, lead author of the study and assistant professor in the Bloomberg School's Department of Biostatistics.
Authors of "Cooperation Between Referee and Authors Increases Peer Review Accuracy" are Jeffery T. Leek, Margaret A. Taub and Fernado J. Pineda.
Funding for the study was provided through a Johns Hopkins Faculty Innovation Award.
No entries found