Communications of the ACM
DARPA Wants to Solve Science's Replication Crisis With Robots
At the same instant that a significant chunk of policymakers seem to disbelieve the science behind global warming, a bunch of scientists come along and point out that vast swaths of the social sciences don't stand up to scrutiny. They don't replicate—which is to say, if someone else does the same experiment, they get different (often contradictory) results.
Researchers are trying to fix the problem. They're encouraging more sharing of data sets and urging each other to preregister their hypotheses. The idea is to cut down on the statistical shenanigans and memory-holing of negative results that got the field into this mess.
And self-appointed teams are even going back through old work, manually, to see what holds up and what doesn't. That means doing the same experiment again, or trying to expand it to see if the effect generalizes. To the Defense Advanced Research Projects Agency, the Pentagon's mad-science wing, the problem demands an obvious solution: Robots.
A DARPA program called Systematizing Confidence in Open Research and Evidence (SCORE) aims to assign a "credibility score" to research findings in the social and behavioral sciences, a set of related fields to which the reproducibility crisis has been particularly unkind. DARPA has promised $7.6 million to the Center for Open Science, a nonprofit organization that's leading the charge for reproducibility. COS is going to aggregate a database of 30,000 claims from the social sciences.
From Wired
View Full Article – May Require Registration
No entries found