Communications of the ACM
Communications of the ACM
Are You Talking to Me?
I recently attended a rather theoretical computer-science conference, and sat, as is my habit, in the front row. The speaker was trying to convey the fine details of a rather intricate mathematical construction. I was hopelessly lost. At that point I found the talk indistinguishable from Doug Zongker's celebrated "Chicken Chicken Chicken" talk presented at the 2007 AAAS Humor Session (http://www.youtube.com/watch?v=yL_-1d9OSdk). Looking behind me to see how other attendees were reacting to the highly dense presentation, I was greeted by a wall of laptop screens; people were busily reading their email.
At the business meeting that evening, I asked "How many people could follow 100% of 100% of the talks?" Silence. "80% of 80%?" One brave soul responded positively. It was only when I got to "50% of 50%" that about 50% of the participants raised their hands. Of course, this statistic should not be taken too seriously, but, nevertheless, I found it shocking! About 100 people are spending four days attending talks, and only 50% understand 50% of 50% the talks? What is the point of this futile exercise?
I am reminded of Lance Fortnow's pithy description of a computer-science conference as "a journal that meets at a hotel." Indeed, if the point of the conference is simply to score a prestigious publication, then attending the conference and giving a talk is just a hurdle that one must overcome as a condition of publication. As I pointed out in my May 2011 editorial, "Technology Has Social Consequences," many conferences eliminated face-to-face program-committee meetings in the late 1990s to save travel expenses and hassle. Why don't we take the next logical step and virtualize our conferences in the name of efficiency?
I am not serious, of course. I actually like conferences very much. I believe they are a critical component of the scientific enterprise. Science is a social undertaking. For most of us, our scientific social network is truly global. Meeting at conferences is the only way to maintain our links, learn what is happening, and tell others about our latest and greatest. While some of the activity of a conference happens in coffee breaks and hallways, its core activity takes place in the lecture halls, and this activity better be effective, which means the talks better be clear, informative, and interesting. Why is it then that we put so much attention on ensuring the quality of the papers, and so little attention on ensuring the quality of the talks?
There are many ways in which we can attempt to improve the quality of conference talks. Some of these measures are easy and obvious. For example, graduate students should be taught that preparing a good talk is quite different from, though equally important as, writing a good paper. They should never give a conference talk without some dry runs with brutally honest feedback from their advisor and fellow students. Also, for their first few conference talks, graduate students should be video-recorded. Many will be rather shocked when seeing and hearing themselves for the first time. This advice applies not only to graduate students. While students often make the rookie mistake of trying to tell the audience everything in their paper, rather than tell the audience about their paper, they are not the only ones giving poor talks.
Conferences should, in my opinion, take active measures to improve presentation quality. A radical proposal would be to require authors to submit not only papers but also video recordings of their talks. The quality of those presentations would be considered in making program decisions. Less radical a move is to require authors to send draft presentations before the conference, and receive feedback from their session chairs. It should also be relatively easy to augment conference-management systems with feedback pages where conference participants can give speakers anonymous feedback on their presentations. (That would give attendees something constructive to do during poor presentations!)
At some conferences, I have raised the issue of poor presentations, and encountered unwillingness by conference officials to take any concrete measure. I am told my proposals are "too intrusive," which is truly puzzling. We manage conference programs with an iron hand, often ruffling many feathers by (sometimes controversial) program decisions. Why are we suddenly "kinder and gentler" when it comes to presentation quality? If conferences are important, then we ought to treat them as more than "journals meeting at hotels" and make sure the time we spend attending them is well spent.
Moshe Y. Vardi, EDITOR-IN-CHIEF
©2011 ACM 0001-0782/11/0900 $10.00
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.
Comments
Moshe Vardi
See http://ubiquity.acm.org/article.cfm?id=1972563
Anonymous
Wow, totally agree.
Btw: you bring up a good point here for further research. It is my firm belief that there will be an almost perfect inverse correlation between the number of people in the audience reading their email and the rating of the speaker by evaluation forms. Once we have established this we can get rid of these eval forms and just count the ratio of open laptops.
Raghavendra Kattinakere
Oh! Was there a face-to-face programme committee meeting before ?
Anonymous
50% of the people understood 50% of 50%. But maybe 60% understood 70% of 30% of the talks, which is not that bad.
Anonymous
The laptop issue is real problem, but if we can't be honest then we can't fix the problem. Presentation skills need to be a consideration. There is one computer scientist that I really admire and try to follow his work, but his presnentations often consist of showing the PDF of the paper being presented. Everyone tunes out after 5 minutes.
Many talks by PhD students are just horrible. Especially the Asian students that lack sufficient English skills to give a talk. I've attended lots of these that are nothing but painful to sit through. No one ever says anything to the student or advisor. If they do the response is often indignation and some response about the critic not being sufficiently "multicultural".
Personally I would rather accept a paper and have the presentation an option. Someone with a history of really bad presentations can be told they don't get to present unless they demonstrate improvement similarly professors are told if they send a student that isn't prepared to present then only the paper will be accepted. An alternate solution would be to only publish papers if the presentation is sufficiently good.
Frank Olken
I mostly agree with these remarks. In particular, I have attended many recent conference talks in large lecture halls (300-500 persons) where the speaker is using 10 point fonts! I have complained to little avail. Also, speakers need to speak into the microphones (usually that means facing the audience not the screen).
Anonymous
I have to admit that I attended some bad talks given by native speakers, but English as a second language is more often a barrier to brilliant lectures. If --and I agree-- science is a social undertaking, the linguistic domination of English is a real problem, especially on the spoken side, with its /analgorithmic/ phonetics especially hard for Asian and Latin people. Rem tene, verba sequentur? Maybe when one can think before writing, not in the interactive setting of a talk, often with half-understood questions.
Anonymous
Moshe points out that a lot of the important stuff happens outside of the presentation. Is the problem not so much a problem with bad presentations but rather a problem with the structure of conferences? Presentations can be made available either in written form or on Utube before and attendees can look at them in their own time. Why waste valuable discussion time? It would be much more valuable to all those interested in the paper and in particular the author if instead of a talk, one had a facilitated discussion on the paper and how it relates to what others are doing? Authors could also be given the opportunity to improve on their papers taking into account comments maked in the discussion. Another idea would be to have mechanisms to arrange meetings for attendees even if they are not giving papers.
Toby Walsh
Another (partial) fix is to raise the profile of poster presentations. Some of the problems with talks are fixed with a good poster presentation (language skills are not so crucial, font sizes less critical, questions can be asked interactively to clarify areas of confusion, ...). However, posters are too often viewed as a second rate acceptance. Conferences like AAAI and IJCAI are fighting this by giving the best papers both a talk and a poster. Other conferences like UAI had the foresight to fix this from the start.
Anonymous
I think the simplest solution is to:
1. Enforce that 80% of the talk is about the why is the topic important and what led to this conclusion.
2. Limit the number of words per slides.
3. Maximize the number of interactive diagrams.
Lot of experience is needed to give a good talk about the optimality of ABC under bla bla conditions. But far less experience is needed if a story is told that what led the author to optimality of ... and why is it useful.