Communications of the ACM
Viewpoint
Paper Presentation at Conferences: Time For a Reset
Credit: Halfpoint
The importance of conferences as a publication medium is now well established in computer science. The methodology of conference reviewing and its role in selecting high quality archival publications has been a subject of much recent discussion. However, there has been little discussion of the role of paper presentation at a conference. This Viewpoint explores this issue and describes an experiment we ran at the VLDB conference.
The traditional paper presentation, at least in the conferences I go to, is 20-plus minutes in duration, most of which time is spent going over details more completely stated in the written paper. These details are also more efficiently understood from the written paper for many of us, who can read technical material faster than we can listen to it. Some attendees may hope for a more in-depth perspective through attending a session. However, when interesting questions do arise, there is usually limited time for discussion. Many would say the goal of a conference presentation should be to convince the audience to read the paper, not to teach them the contents of the paper. But this limited goal can probably be accomplished almost as well in much less time. Consequently, I personally see little value in attending a traditional research talk at a conference.
A remedy that has recently become popular in some conferences is to have a poster session for all papers. This provides an opportunity for the in-depth personal interaction that paper sessions do not provide. Additionally, it is an opportunity for an attendee to browse quickly through many papers, sometimes complemented with a gong-show for this purpose.
With this avenue for in-depth interaction in place, the traditional research paper session is no longer even pretending to fill this role. Indeed, some conferences have gone so far as to restrict oral presentation to only selected papers.
So what should the role of a session be? I think it is self-evident that an interesting session must be conceived and planned as a session on some topic, rather than as a mere union of independent paper presentations. If our focus is on an individual paper, we can do much better by reading the paper and going to a poster session. The reason to go to a research session is to benefit from the presence in one room of multiple experts interested in some topic (or, at least, closely related topics). The question becomes how best to accomplish this.
In some fields, conferences are organized by session, and papers are invited to particular sessions. Such conferences find it easy to have cohesive sessions. But this flexibility is not available to most computing conferences, which have a carefully devised review process for paper acceptance. In short, in putting a conference program together, we get input at the unit of papers but must produce output at the unit of sessions. This is difficult, but not impossible.
I was recently program chair of the VLDB conference, and this gave me an opportunity to try some things out. So let me describe a few of the things we did, and how it turned out (see http://www.vldb.org/2014/program/Menu.html). My evaluation is based on both anecdotal evidence and the results from an attendee survey we conducted.
We asked session chairs (of research paper sessions) to present an overview of the research sub-area, we restricted paper presentations to 12 minutes, and we asked questions to be mostly deferred to a mini-panel at the end of the session. Since this was the first time we were trying these changes, session chairs were given considerable leeway in how they implemented these changes, and were encouraged to be creative in building an interesting session. Thus we had 32 uncontrolled experiments, one per research paper session, and report here on the "results."
The reduction in paper-presentation time to 12 minutes was initially resisted by many authors, who were appropriately worried they would not be able to meet the traditional expectations of a conference presentation within the limited time. It was surprisingly easy to modify expectations, attendees mostly loved it, and the shorter presentations were a resounding success. In fact, there may be room to shorten it further, perhaps to 10 minutes per paper presentation. By so doing, we make time for the more interesting parts of the session, described next.
The session chair introductions varied greatly in style. Even their length varied, from under five minutes to more than 15. The response to session chair introductions was generally positive, and most attendees felt it really helped them understand the big picture before diving into the weeds with individual papers. Based on feedback, my strong recommendation is the introductions try to present an overview of the research frontier in the sub-discipline, with a brief mention of where each paper fits in this scheme. The actual contributions of each paper, and the related earlier work that each depends on, are best left to the individual presentation.
The mini-panel at the end had somewhat mixed reactions but was positively received on balance. The two most salient criticisms were: the authors did not engage in discussion with one another and there seemed to be a strong recency effect, resulting in more audience questions addressed to talks later in the session. To counterbalance, an equally common "criticism" was there was insufficient time allotted to the most enjoyable part of the session. If I were doing this again, there are a few things I would do slightly differently. First, I would explicitly ask each author to read the other papers in the session and come prepared with one question they would like to discuss with each of the authors. In contrast, at VLDB 2014, we only asked authors to communicate with session chairs, and vice versa, ahead of time. Second, I would explicitly ask the session chair to initiate and manage a discussion, selecting at least the first couple of topics to be discussed. At VLDB 2014, our instructions to session chairs were much weaker, merely asking them to come prepared with some questions they could ask the panel if the audience was too passive: in effect, asking them to provide a back-up rather than drive the intellectual stimulation.
As program chair, I spent a great deal of time coming up with a good first cut that partitioned papers into sessions.
How closely related the papers were in a session did appear to matter. In particular, it was evident from the start that we could not possibly have any "potpourri" sessions. To have some flexibility in session allocation, we constrained sessions to have four, five, or six papers. (In a later phase of program construction, sessions with the same number of papers were placed in parallel, to the extent possible).
As program chair, I spent a great deal of time coming up with a good first cut that partitioned papers into sessions. Then, I broadcast this tentative design to all authors, and got some good feedback. Many authors simply requested their own paper be moved to a different session. But many others had more ambitious suggestions, including two suggestions for completely new sessions (of which I was able to adopt one). In short, the authors and session chairs have a vested interest in at least local pieces of the conference organization. Getting them involved before setting things in stone was really helpful.
Even with all care and effort, some sessions were more cohesive than others. This probably cannot be helped. However, there was no session where the session chair complained about a specific paper being too far of a reach for the session.
Finally, and completely unrelated to the aforementioned changes, we decided to recognize excellence in conference presentation. We deputized the 32 session chairs to nominate candidates for this (non-competitive) certificate, not necessarily from their own sessions. We received 13 nominations, including two from one session chair. This number is well under 10% of the (165) papers presented at the conference. Each of the 13 presenters was recognized with a certificate, and a mention on the conference website. Since this award was specifically for the presentation, only the presenters were recognized and not their co-authors.
We had an active email discussion among session chairs before, during, and after the conference. One issue that concerned many session chairs was the ability of a conference attendee to switch sessions to listen to specific paper presentations. This turned out to be a non-issue in practice. Where people had particular papers of interest, they met the author(s) at the poster session, and there was very little evidence of session hopping for particular paper presentations. Philosophically, I believe the right way to think about the issue is that the paper is the unit of acceptance (and poster presentation) but the session is the unit of conference organization. If attendees want a good conference experience, we need to be thinking about the experience in larger rather than smaller units. I think there is much that conference chairs can do to make research paper sessions first-class entities in the conference program, and we should insist they do, so that conferences remain intellectually stimulating events, and not become just networking venues. For additional details, see "Real Session Chairs Do More Than Just Keep Time" at http://www.vldb.org/pvldb/vol7/FrontMatterVol7No12.pdf.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.
No entries found