Communications of the ACM
(Re)Use of Research Results (Is Rampant)
Credit: Getty Images
According to Popper,23 the ideas we can most trust are those that have been the most tried and tested. For that reason, many of us are involved in this process called "science," which produces trusted knowledge by sharing one's ideas and trying out and testing the ideas of others. Science and scientists form communities where people do each other the courtesy of curating, clarifying, critiquing, and improving a large pool of ideas.
Key Insights
According to this definition, one measure of a scientific community's health is how much it reuses results. By that measure, the software engineering research community might seem to be very unhealthy. Da Silva et al. reported that from 1994 to 2010, only 72 studies had been replicated by 96 new studies.10 In February 2022, as a double-check for da Silva's conclusion, we queried the ACM Portal for products from the International Conference on Software Engineering (ICSE), that field's premier conference. Between 2011 and 2021, only 111 out of the 8,774 ICSE research entries were labeled as 'available,' 74 as 'reusable,' 24 as 'functional,' and none as 'replicated' or 'reproduced' reuse (see Table 1 for a definition of those terms). Put another way, according to the ACM Portal, only 2.4% of the ICSE publications are explicitly associated with any kind of reuse. Worse still, according to that report, there were no replicated or reproduced results from ICSE in the last decade.
No entries found