Communications of the ACM
Are You Talking to Me?
I recently attended a rather theoretical computer-science conference, and sat, as is my habit, in the front row. The speaker was trying to convey the fine details of a rather intricate mathematical construction. I was hopelessly lost. At that point I found the talk indistinguishable from Doug Zongker's celebrated "Chicken Chicken Chicken" talk presented at the 2007 AAAS Humor Session (http://www.youtube.com/watch?v=yL_-1d9OSdk). Looking behind me to see how other attendees were reacting to the highly dense presentation, I was greeted by a wall of laptop screens; people were busily reading their email.
At the business meeting that evening, I asked "How many people could follow 100% of 100% of the talks?" Silence. "80% of 80%?" One brave soul responded positively. It was only when I got to "50% of 50%" that about 50% of the participants raised their hands. Of course, this statistic should not be taken too seriously, but, nevertheless, I found it shocking! About 100 people are spending four days attending talks, and only 50% understand 50% of 50% the talks? What is the point of this futile exercise?
I am reminded of Lance Fortnow's pithy description of a computer-science conference as "a journal that meets at a hotel." Indeed, if the point of the conference is simply to score a prestigious publication, then attending the conference and giving a talk is just a hurdle that one must overcome as a condition of publication. As I pointed out in my May 2011 editorial, "Technology Has Social Consequences," many conferences eliminated face-to-face program-committee meetings in the late 1990s to save travel expenses and hassle. Why don't we take the next logical step and virtualize our conferences in the name of efficiency?
I am not serious, of course. I actually like conferences very much. I believe they are a critical component of the scientific enterprise. Science is a social undertaking. For most of us, our scientific social network is truly global. Meeting at conferences is the only way to maintain our links, learn what is happening, and tell others about our latest and greatest. While some of the activity of a conference happens in coffee breaks and hallways, its core activity takes place in the lecture halls, and this activity better be effective, which means the talks better be clear, informative, and interesting. Why is it then that we put so much attention on ensuring the quality of the papers, and so little attention on ensuring the quality of the talks?
There are many ways in which we can attempt to improve the quality of conference talks. Some of these measures are easy and obvious. For example, graduate students should be taught that preparing a good talk is quite different from, though equally important as, writing a good paper. They should never give a conference talk without some dry runs with brutally honest feedback from their advisor and fellow students. Also, for their first few conference talks, graduate students should be video-recorded. Many will be rather shocked when seeing and hearing themselves for the first time. This advice applies not only to graduate students. While students often make the rookie mistake of trying to tell the audience everything in their paper, rather than tell the audience about their paper, they are not the only ones giving poor talks.
Conferences should, in my opinion, take active measures to improve presentation quality. A radical proposal would be to require authors to submit not only papers but also video recordings of their talks. The quality of those presentations would be considered in making program decisions. Less radical a move is to require authors to send draft presentations before the conference, and receive feedback from their session chairs. It should also be relatively easy to augment conference-management systems with feedback pages where conference participants can give speakers anonymous feedback on their presentations. (That would give attendees something constructive to do during poor presentations!)
At some conferences, I have raised the issue of poor presentations, and encountered unwillingness by conference officials to take any concrete measure. I am told my proposals are "too intrusive," which is truly puzzling. We manage conference programs with an iron hand, often ruffling many feathers by (sometimes controversial) program decisions. Why are we suddenly "kinder and gentler" when it comes to presentation quality? If conferences are important, then we ought to treat them as more than "journals meeting at hotels" and make sure the time we spend attending them is well spent.
Moshe Y. Vardi, EDITOR-IN-CHIEF
©2011 ACM 0001-0782/11/0900 $10.00
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.
Comments
Anonymous
Half the folks understanding half of half the talks is not bad. I think it's much worse than that. I suggest the following.
- Shorter talks (10 mins + 2 questions at most).
- Make poster sessions the main feature. Everyone brings a poster as well.
- Forbid open laptops during presentations.
Amit Chopra
Anonymous
Couldn't agree more - most presenters don't know how to inspire, educate, or entertain (yes, entertain). The presentation is a mere formality to get their work "on the record".
It's interesting, though, that you mix audience comprehension and presenter quality in the same article from which I infer they are related. I would propose they are not. One can give a great presentation, but if the audience hasn't been working in that particular field it may be incomprehensible. Or my presentation may be dull as a stack of cord wood but completely understandable by virtually everyone.
Perhaps we're asking too much. Engineers and computer scientists can barely write. Should we expect them to talk too ;-)
Anonymous
What about simple motivating actions, like inviting great speakers, or recording all talks then publishing them online (shaming the worst speakers into improving their presentations) or by giving the best talks more exposure?
For instance, at the conferences organized by ESUG.org, the audience is mixed industrial / opensource / academic, and we were lucky to have a few members of the community with who were really good speakers, so they set the example and encouraged everyone. In just a few years the quality of talks as greatly improved.
Anonymous
Moshe,
I read your letter in the September issue of Communications of ACM with great interest.
If you changed your question slightly and asked if I understood 50% of 50% of the content of CACM I would not be able to say yes. The Communications does not seem to communicate, at least in my case. Using your terms maybe "Communications" should be changed to "Journal". If I may quote you: "What is the point of this futile exercise?"
Thanks,
Bill Olmo
Anonymous
Reviews for slides may sound like a good idea. However, in my experience it is already difficult to find enough reviewers with enough time to review just the papers! Besides the technical hurdles for the author to prepare a quality video recording, this will also require even more time from the reviewers, and will be very difficult to comment on without something like a written transcript to go along with it. Who will prepare the transcript? Is is accurate?
Finally, it's important to note that it would be a mistake to include the presentation in judging the quality of the paper. Some people are just not good at presentations. Those people should not be rejected from journals because of that. And due to anonymity concerns, the video would be hidden from reviewers until after the paper decision anyway.
In summary, it's a good idea if you have plenty of reviewer resources to burn and your authors have a good A/V staff available to them. I think both of these are rarely the case.
Moshe Vardi
Communications of the ACM is a magazine. The word magazine comes from the Arabic word makhzan, which means storehouse. A magazine is a publication that contains a collection of articles of different types. Communications contains News, Viewpoint, Practice, Contributed, Review, and Research Highlight articles. There is no expectation that readers will read the magazine cover to cover. The expectation is that every reader will find some articles of interest. It is easy to skip an article or abandon it after a page or too. It is much more difficult to leave a talk after it has started.
Henry Ledgard
This is a wonderful article. It is spot on.
The same thing happens at (I think) every graduate program, where lectures (faculty too) and student thesis defenses are out of touch with the audience. The suggestions about dry runs and video recording are excellent.
It is so nice that someone has spoken up !
The account that made this comment no longer exists.
People should be careful when reviewing draft presentations. As we know, or imagine, good presentations have simple, clear slides, which the speaker *complements* with content, never just reading bullet points, for example.
If you have a really good presentation and send the PDF with the (very simple) slides for peer-review, the reviewer might not even understand the idea of the talk or the connection with the paper, and may reject it based on "poor quality".
A presentation is never just the talk, nor is just the slides. So yes, either authors send a video of previous presentations or they send nothing and we all keep our fingers crossed.
Anonymous
Every conference fee should include the costs of this excellent guide: http://www.thinkoutsidetheslide.com/vsr.htm
Anonymous
People should realize that a presentation is not a karaoke: in general, you should not put on a slide what you are saying, but what you cannot say
Displaying comments 11 - 20 of 28 in total