Communications of the ACM
New Paradigm For Scientific Publication and Peer Review
Credit: iStockPhoto.com
Scientists spend too much of their time publishing papers and ploughing through the mountains of papers produced by their colleagues, and not enough time doing science.
That's the observation—and frustration—that spurred Fabio Casati and his collaborators to launch LiquidPublication, an EU-financed research project that seeks to revolutionize how scientists share their work and evaluate the contributions of their peers.
"The more papers you produce, the more brownie points you get," says Casati. "So most of your time is spent writing papers instead of thinking or doing science."
Besides wasting untold hours, Casati says, the current scientific publication paradigm produces other toxic fallout including an unduly heavy load for peer reviewers and too many papers that recycle already published research or dribble out results a bit at a time.
"The current system generates a tremendous amount of noise," he says. "It's hard to find interesting new knowledge because there's so much to see."
Casati and his colleagues are developing and promoting a radically new way to share scientific knowledge, which they call "liquid publication." They want to tap the power of the Web—including its ability to speed communication, facilitate data storage, search and retrieval, and foster communities of interest—to replace traditional peer reviews and paper publications with a faster, fairer and more flexible process.
"If we can make scientists' work even 10 percent more efficiently, it will give a great benefit to the community," Casati says.
Don't Print It; Post It
Following the lead of physicists and mathematicians who for years have been posting early versions of their papers on a website called arXiv.com for quick dissemination and peer critiques, Casati and his colleagues propose that all scientists jump start the dissemination of their findings by posting them online.
"The idea is that when people write papers, they put them on their webpage quickly, easily and for zero cost," Casati says.
At the same time, Casati suggests, every scientist and research group can create its own "liquid journal" which groups publications that are interesting and relevant to a given topic.
"Suppose I want to create a journal on, say, interesting findings in peer review, that I want to use to collaborate with my research group and my peers," says Casati. "I will go fish for interesting papers that are on the Web. People don't submit to my journal, they just post to their webpage or to an archive. I find the paper and include it in my journal. Everybody can do this."
Furthermore, a journal need not contain only papers. "Experiments, datasets, and even blogs can be first-class citizens in the work of science," he says.
Readers of a liquid journal can 'create' knowledge too. Besides accessing or commenting on papers, readers can link papers (for example to point out that paper P2 is an evolution of paper P1, or to note that paper P reports on a given experiment which is, in turn, performed on a certain dataset). Such contextual knowledge is essential to finding information easily, Casati suggests.
Another benefit Casati foresees from liquid publication is a reduction in multiple papers that merely report incremental new results. Instead, borrowing a tool from software developers, he would like to see such incremental changes clearly identified by versions—for example "beyond peer review" versions 1, 2 and 3.
"Instead of completely different papers, version 3 would appear as a small adjustment to version 2," Casati says. "When I as a reader search for knowledge, I can go directly to the most recent version. And this 'connection knowledge' is provided for free by the community."
Don't Review It; Use It
This radical new approach to scientific publication offers an equally radical alternative to the peer review process.
For the past 300 years, Casati argues, printing and publishing a scientific paper was a costly process. Because of this, gatekeepers were needed to judge which contributions were worth publishing; hence peer reviews.
Since liquid publications cost nothing, he says, a major justification for those gatekeepers vanishes.
But what about quality control, the important task of shepherding good research into the limelight and blocking sloppy or even falsified findings from the research canon?
"We've studied this and found that peer review doesn't work, in the sense that there seems to be very little correlation between the judgment of peer reviewers and the fate of a paper after publication," says Casati. "Many papers get very high marks from their peer reviewers but have little effect on the field. And on the other hand, many papers get average ratings but have a big impact."
Casati and his colleagues suggest replacing peer review—in which typically three researchers determine the fate of a piece of research—with the assessment that is implicitly given by the relevant community while editing and reading liquid journals.
"If you and I include this paper in our journals, we are giving it value," says Casati. "When this is done by hundreds of people like us, we're using the selection power of the entire community to value the contribution. Interesting papers will rise above the noise."
Scientists already supplement peer review with measures of how frequently a paper is cited by other papers. Casati believes that the community evaluation approach would prove more immune to distortions, for example by researchers pushing their findings or authors citing papers they haven't bothered to read.
A further bonus of liquid publication, Casati says, would be to give appropriate value, for the first time, to contributions such as a stimulating blog, a nice data set or a useful computer code. "With liquid publication and community evaluation," says Casati, "everything counts."
The LiquidPublication consortium is putting its ideas into practice, starting with an open source software platform and its own liquid journal on peer review.
"We're bootstrapping this approach," says Casati. "We believe that people will start doing this when they find that it's useful to them, like Google or Facebook."
They may be starting small, but the interest of consortium members, such as the French National Centre for Scientific Research and the publishing powerhouse Springer Science, reflects the fact that they are tackling a large and extremely important problem.
The LiquidPublication project received funding from the FET-Open strand of the Seventh Framework Program for research.
From ICT Results
No entries found