Communications of the ACM
Home/Magazine Archive/March 2010 (Vol. 53, No. 3)/Revisiting the Publication Culture in Computing Research/Full Text
Communications of the ACM
Revisiting the Publication Culture in Computing Research
In my May 2009 Editor's Letter, "Conferences vs. Journals in Computing Research" (p. 5), I addressed the publication culture of our field: "As far as I know, we are the only scientific community that considers conference publication as the primary means of publishing our research results. In contrast, the prevailing academic standard of 'publish' is 'publish in archival journals.' Why are we the only discipline driving on the conference side of the 'publication road?'"
In response to my editorial, Lance Fortnow wrote a Viewpoint column (Aug. 2009, p. 33), entitled "Time for Computer Science to Grow Up," in which he concluded: "Computer science has grown to become a mature field where no major university can survive without a strong CS department. It is time for computer science to grow up and publish in a way that represents the major discipline it has become."
The May 2009 editorial and the August 2009 column attracted a lot of attention in the blogosphere. The reaction has been mostly sympathetic to the point of view reflected in both pieces. For example, Jeanette Wing asked in her blog: "How can we break the cycle of deadline-driven research?", and Filippo Menczer, in a Letter to the Editor published in the November 2009 issue, said: "I propose the abolition of conference proceedings altogether."
Not everyone, however, agreed with this point of view. For example, in another Letter to the Editor from the November 2009 issue, Jano van Hemert said: "For CS to grow up, CS journals must grow up first." Mr. van Hemert's issue with computing-research journals is that they are known to have "slow turnaround, with most taking at least a year to make a publish/reject decision and some taking much longer before publishing." Such end-to-end times, he argued, "are unheard of in other fields where journal editors make decisions in weeks, sometimes days."
While I have not see concrete data comparing publishing turnaround times for computing-research journals to those in other technical fields, there is abundance of anecdotal data supporting the claim that computing-research journals are indeed quite slow. (The average time to editorial decision for Communications is under two months; that takes a concerted effort by the editorial board to ensure that the editorial process does not stall.)
What is the reason for the unacceptably slow turnaround time in computing-research journals? When considering this question, one must factor the problem into two separate issues: time from submission to editorial decision, and time from positive editorial decision to publication.
First let us address the latter issue. All periodical journals have editorial "pipelines." No publisher wants to face the threat of an empty issue; it's akin to the dreaded dead air on television! Successful journals that attract many submissions often see their pipelines extend for up to two years. With the advent of electronic publishing, this problem can be eliminated or at least minimized. Communications uses its Virtual Extension (VE) to ensure its pipeline does not get longer than six months. VE articles undergo the same rigorous review process as those in the print edition and are accepted for publication on their merit. These articles are available in ACM's Digital Library.
Let us now consider the editorial process in computing-research journals. Why is it soooo slow? Consider who is in charge of that process. It is not the publishers; it is the editors and referees. In other words, it is us. The process is slow because that is the way we run it. If we want it changed, it is up to us to change it! I suspect that we cannot separate our conference-focused publication culture from our sluggish journal editorial process. Conferences have sharp deadlines, journals do not. We simply do not take our roles as editors and referees as seriously as we do as program committee members because we do not take journals as seriously as other fields. If we, as a community, decide that we need to shift from conference-based publication to journal-based publication, we definitely must address the slow editorial process, but we should not complain about "them journals." We have found the enemy, and it is us!
The 2010 Conference of the Computing Research Association (July 1820, Snowbird, UT) will have a plenary panel on "Peer Review in Computing Research." I look forward to that discussion and hope it will help our community reach consensus on this issue.
Moshe Y. Vardi, EDITOR-IN-CHIEF
©2010 ACM 0001-0782/10/0300 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.
Comments
Abazer Ali
There is no doubt that computer science has become as significant Magazines, ranging from areas such as databases and web pages as well as computer networks. I agree with the statement "in order to grow up computer science has to grow up scientific journals.
Another point I want to acknowledge it, a very careful before publishing houses in the adoption of patrols in computer science.
Because I asked one of them days in the book fair, why he wrote the computer publishes are part of the lowest among the headings in the show?
He replied that this area is progressing rapidly enormous which we publish today may become undesirable in a year or less.
Accordingly, I believe that the process of transformation of conferences and publication to the journals require a lot of reflection and consideration in resolving the dilemmas faced by computer science before sagging.
With my sincere thanks and respect
Sameer Abufardeh
Yes, the editorial process in computing-research journals is very slow and somewhat frustrating. My own experience taught me that to prepare a paper for a journal I will need at least twice the time needed for a conference, twice the wait, etc. For a graduate student (MS, PhD) who is expected to publish some of his work before graduating, the long wait, the endless formatting requirements by journals, can be time consuming, overwhelming, and somewhat frustrating. Furthermore, I dont believe that there is any tangible quality gap between conference paper and journal paper. Conference papers are generally short and right to the point because of the limitations normally set forth on the # of pages allowed. - Kind Regards to all
CACM Administrator
The following letter was published in the Letters to the Editor in the July 2010 CACM (http://cacm.acm.org/magazines/2010/7/95049).
--CACM Administrator
In his Editor's Letter "Revisiting the Publication Culture in Computing Research" (Mar. 2010), Moshe Y. Vardi said computer science is "the only scientific community that considers conference publications as the primary means of publishing our research results," asking, "Why are we the only discipline driving on the conference side of the 'publication road?'"
As an old timer, I can say that in the early days, there was a belief (conceit might be a better word) that the field's pace of discovery was happening so quickly that only conferences, with subsequent prompt publication of proceedings, could communicate results in a timely manner. As a corollary, the traditional peer-reviewed published literature review fell behind, as it was relieved of temporal pressure through the published proceedings.
These days, the pace of discovery in the biological sciences, including molecular biology, genomics, and proteomics, far exceeds that of computer science. Yet the gold standard of publication in archival journals continues. It is the ultimate irony that computer science, along with various disciplines in the physical sciences, employs the tools developed by computer scientists to ensure timely dissemination of research results through the online editions of their publications. Science, Nature, Cell, and other leading journals routinely present their most important articles in online form first. If, perhaps, computer science would make greater use of its own tools, the shoemaker's children would no longer go barefoot, and published proceedings would fade into its proper historical niche.
Stuart Zimmerman
Houston, TX
CACM Administrator
The following letter was published in the Letters to the Editor in the May 2010 CACM (http://cacm.acm.org/magazines/2010/5/87250).
--CACM Administrator
Moshe Y. Vardi wrote in his Editor's Letter "Revisiting the Publication Culture in Computing Research" (Mar. 2010) about computer science being a field in which conferences are the primary venue for reporting research. Over the years, the premier conferences have evolved to where their acceptance rates are lower than the corresponding journals and their citation rates are higher. That is why, in the university, the other fields in the tenure councils accept the computer science argument that conference publications deserve the main weight.
When I first got involved in the ACM publication establishment (late 1960s) there was a terrible problem with backlogs in the two ACM research journals of the day: Communications and the Journal of the ACM. Editorial time submission to final decision was running 1218 months, and publication time decision to publication took at least as long. Many loud complaints agreed that 2436 months for the complete process was bad for such a fast-changing field. However, ACM was almost broke and could not afford more pages, and there was no online venue. Researchers began to look to the SIG conferences as a way to get their material into print sooner, much sooner. By the 1980s, some of these conferences were petitioning the Publication Board to designate them as "refereed" instead of "reviewed."
Studies by Bob Ashenhurst and John Rice documented the editorial delays. Even when editors were fastidious about pushing the review cycle along, the authors often took a long time to submit their revisions. Though individual reviews took a long time as well, the authors themselves contributed a lot of delay. The studies also found (with much less supporting data) that authors try different journals and conferences until they find a publisher, and that patient, persistent authors eventually publish over 90% of their papers. Rice calculated an average paper needs four referees, so authors "owe" their colleagues four reviews for every published paper.
A productive author publishing three journal papers a year would thus owe the field 12 reviews a year. Most researchers complain if they have to do half that number. Rice's conclusion, still valid today, was that as long as researchers do not want to do all those (timely) reviews, the editorial phase would not get significantly shorter and is simply the cost of an all-volunteer system.
In 1983 at the start of the Communications revitalization plan, we investigated how Science magazine gets its submissions reviewed so much quicker. We visited the editors and learned they phoned names from a reviewer database to ask for two-week turnaround. After locating three agreeable reviewers, the editors would FedEx them the manuscript.
We wanted to do likewise, but the ACM executive committee said it was way too expensive. At the time, ACM didn't have enough money to cover even the editors the Council had approved and could not mount a quick review process in Communications. Today's online tools make it much easier and less expensive to find reviewers, in which case the bottleneck is again the willingness of reviewers to quickly review and authors to quickly revise.
Peter J. Denning
ACM Past President, 19801982
Monterey, CA
CACM Administrator
The following letter was published in the Letters to the Editor in the May 2010 CACM (http://cacm.acm.org/magazines/2010/5/87250).
--CACM Administrator
A fundamental difference between the journal and conference publication systems, as discussed by Moshe Y. Vardi (Mar. 2010), is that conference leadership (chairs and program committees) changes much more frequently than (is practical) for journals. It is difficult (though perhaps not impossible) for a conference to consistently reflect the personal biases and hidden agendas of the same people over a long period of time. The same cannot be said of journals, which have much slower turnover of editors. Such stability allows journals to be, at least in principle, more coherent and less prone to fads, and perhaps less responsive to what the community views as important.
Angelos D. Keromytis
New York
Displaying all 5 comments