Communications of the ACM
Holding a Conference Online and Live, Due to COVID-19
The joint EDBT/ICDT conference (International Conference on Extending Database Technology/International Conference on Database Theory) is a well established conference series on data management, with annual meetings in the second half of March that attract 250 to 300 delegates. Three weeks before EDBT/ICDT 2020 was planned to take place in Copenhagen, the rapidly developing COVID-19 pandemic led to the decision to cancel the face-to-face event. In the interest of the research community, it was decided to move the conference online while trying to preserve as much of the real-life experience as possible. As far as we know, we are one of the first conferences that moved to a fully synchronous online experience due to the COVID-19 outbreak. By fully synchronous, we mean that participants jointly listened to presentations, had live Q&A, and attended other live events associated with the conference. In this report, we share our decisions, experiences, and lessons learned.
1 Introduction
Three weeks before EDBT/ICDT 2020 was planned to take place in Copenhagen, the rapidly developing COVID-19 pandemic reached a state in which it became clear that the face-to-face event had to be canceled. We, the organizers, then decided to move the conference online while trying to preserve as much of the real-life experience as possible. Given the very short notice, we had to be pragmatic and could not prepare the online event as carefully as we would have done otherwise. In fact, we considered the whole thing an interesting experiment suggested by the circumstances, with potentially important lessons to be learned for the community and beyond. We were delighted to receive significant support for our decision from the attendees and keynote speakers and by the community spirit that developed in its course. The online event ran smoothly and was much more enjoyable and successful than we had expected. We received a lot of positive feedback, both informally and in the survey that we sent to our participants after the conference. The purpose of this document is to share our experience and the lessons learned, so that other conference organizers facing a similar situation can benefit from it.
2 Decisions
We go over a list of questions that other conference organizers are likely to face when considering a move online, share our decisions, describe how we implemented them in practice, and what we think about the outcome in retrospect. Since the conference needed to be re-organized within a very short time and the effect on internet services of millions of people working from home because of the pandemic was not clear, we used an important guiding principle:
Aim to minimize our dependency on stable Internet connections. (GP)
Should the conference be synchronous (i.e., live) or asynchronous? We decided to run the event in a fully live mode to simulate a face-to-face event as much as possible. Participants jointly listened to presentations, had live Q&A, and attended other live events associated with the conference. This was achieved by combining the Zoom video conference software with the Slack communication platform. Sessions took place on the dates originally planned for the face-to-face event and we used one Zoom meeting for each of the planned sessions, with a password provided to delegates. Fully asynchronous, in contrast, could mean to put videos of the presentations online to be watched by participants at the time that fits them best and to have asynchronous Q&A in Slack or a similar tool.
In retrospect, we are satisfied with choosing the synchronous approach: interaction and discussion are key features of conferences and although these cannot all be fully preserved online, we managed to preserve them to a reasonable extent. Indeed, interesting discussions emerged after many presentations and, to a lesser extent, also in Slack. With more time to prepare and better tool support, we believe that even more interaction can be fostered.
How do we deal with time differences? It seems difficult to deal with time differences when participants are evenly distributed around the globe. In EDBT/ICDT, however, the bulk of participants are from Europe, followed by North and South America, and Asia. To accommodate the relevant time zones, we opted for shorter days, about 5 to 6.5 hours, rather than the expected length (8-9 hours) in a face-to-face meeting. The days were centered around noon CET, which allowed attendees from other continents to attend most sessions without major hassles. For some talks, we made adjustments to the schedule in order to let the speaker present live, e.g. from North America. For example, keynote talks took place at different times on different days as keynote speakers were from different time zones.
How long should sessions and presentations be? Research sessions generally spanned an hour with the net talk length for each papers being 10 and 12 minutes for EDBT and ICDT, respectively. In comparison, EDBT and ICDT allot 20 and 25 minutes for presentations in faceto- face conferences. The main reason for having shorter talks was that we expected longer online sessions to be very tiring for attendees. Another reason is that shorter talks help in achieving shorter conference days to deal with time zone issues as discussed above.
In retrospect, we were happy with the shorter presentations and the sessions didn't seem too long. This impression was confirmed by the participants in our survey (Section 3).
Should talks be recorded or given live? Following our guiding principle (GP), we opted for pre-recorded talks that we then streamed live from a central place with a high capacity internet connection. Our aim was to minimize the probability of technical problems that might result from participants having differing internet connection quality and not being sufficiently familiar with the Zoomsoftware. We suggested to use Zoom to record videos, with the speaker visible, which also helped to get participants acquainted with Zoom. We considered using Microsoft CMT for video upload, but ended up using a simple sFTP solution as CMT has a file size limit of 100MB. We checked the quality of the videos beforehand. We plan to make the videos publicly available after the conference.
Mostly this worked well, with many good quality videos being submitted. On the one hand, pre-recording talks seemed to result in presentations that were well planned, to the point, and with almost no slips of the tongue. On the other hand, the talks were sometimes more monotonous and less dynamic. Some presentations, including 2 keynotes and 1 tutorial were presented live. The keynotes and the tutorial went fluently, but in one of the workshops there were some technical issues with people presenting live. Practice sessions seem necessary for live presentations.
How can questions be managed? Zoom has two modes, the meeting mode and the webinar mode. In the beginning of the conference, we used the webinar mode. This mode has a text-based Q&A facility that allows participants to type their questions and to upvote questions asked by other participants. We then had the people with the most popular questions ask 3 them face-to-face. Some participants asked questions in the Zoom chat, but this rendered the chat (which sometimes uses pop-ups) distracting during presentations. Especially when talks are very short, the smallest distraction can bring listeners off-track. We also generated one Slack channel per session, where question and discussion could continue "offline". Slack was a welcome technological addition, which speakers also used to post their slides after the talk.
The Q&A facility in Zoom is quite good in principle, but it is only available in webinar mode. For smaller, parallel sessions we preferred a more informal approach based on Zoom's meeting mode, which we switched to on the second day. In that mode, participants can see a list of the names of the other participants and they can activate their sound and video (but there is still a meeting host who can mute everyone, e.g., when a talk starts). In meeting mode, we simply asked participants to switch on their camera and raise their hand to indicate that they want to ask a question, as in a real conference. We encouraged people to also switch on their camera after each talk even if they did not want to ask a question, the aim being to create a community feeling, which was quite successful. Many people also switched on their cameras at the beginning and end of each sessions. We still used webinar mode for larger audiences, such as the keynotes.
Retrospectively, we strongly prefer meeting mode and an informal approach to Q&A whenever the audience is of moderate size, say up to 50 participants. This brought much more interaction. Interestingly, quite intense discussions emerged after some talks, probably even more intense than in a face-to-face meeting. This might be due to the group feeling created by Zoom meeting mode when several people have switched on their camera, whereas in a faceto- face event, the few people who are interested in an in-depth discussion of a presentation might sit far apart from each other, with much less of a group feeling.
How can sessions be chaired? Following our guiding principle (GP), we started with only a Zoom host and no session chair. It turned out, however, that the single host is rather busy with running the session, playing the videos, and monitoring the chat. As the event went by, the technology was holding up, and internet connections seemed to be sufficiently stable, session chairs were introduced to manage discussions while a separate Zoom host was technically managing the Zoom session, and this was felt to be successful.
Can there be a social program? The program only had very short coffee breaks of 15 minutes, to make the conference days shorter. There was no joint activity during the coffee breaks apart from using the Slack channels. We held two "Bring your own beer" receptions, one on the opening evening of ICDT, and one on the opening evening of EDBT, where evening refers to the CET time zone. In the receptions, people arrived in an online session and were assigned at random to Zoom breakout groups, to allow smaller group interactions.
Retrospectively, we would go for longer coffee breaks to make the conference days less exhausting. The receptions seemed to work well, given the circumstances, and this was also confirmed in our survey (Section 3). There might well be scope for having more sessions with opportunities for extended, informal, interactions. It might be interesting to use other, technically more sophisticated tools for this, such as Online Town.
How should short/poster papers be handled? We decided to waive short advert videos, and use Slack for asynchronous discussions. We didn't have a clear idea how to run more interactive sessions that would simulate a poster session, so the short paper session turned into a collection of short videos (26 in all) back-to-back with no intermediate Q&A.
The session was well attended, with over 50 people there throughout. However, it was hard work to sit through so many, diverse, videos, and the Slack channel was not especially busy. Retrospectively, we would be tempted to simulate a poster session in a more realistic way to enable deeper interactions. One way of doing this could be to have each poster participant create their own Zoom meeting from within Slack such that participants can use Slack to easily switch between the rooms.
What should be the approach to demos? We adopted the conference model of videos of the demo in 15 minute slots. Each demo was a 10 minute video, with 5 minutes for questions. In retrospect, the videoswere alright, but there wasn't the chance for extended discussions that are associated with demo sessions. Given that technology and internet connections were more stable than we expected, we would be tempted to try having each demo participant create their own Zoom meeting from within Slack such that participants can use Slack to easily switch between the rooms. Again, this more closely reflects the experience at a face-to-face demo session.
What should be the approach to keynotes and tutorials? Keynotes and tutorials were in 1-hour slots, some of them with gaps every 15 minutes for questions. A complete hour of presenting seemed rather long. Furthermore, in an online webinar everyone feels close to the presenter, so some people are more inclined to ask questions. As such, one may want to plan more discussion time.
Two keynotes used videos, two were presented live. Concerning tutorials, three of themused pre-recorded videos, one was presented live. The division of talks into parts was felt to have been a success for both keynotes and tutorials in order to let people chime in with questions.
What do we do if the meeting host has technical problems? Before the conference, we tested how the software platform reacts if the meeting host drops out (e.g., by losing the Internet connection). In our case, we observed that the meeting can still continue, but the host's video freezes. In order to avoid major technical problems, we reached out to back-up hosts for each session, who received a crash course on how to handle the software platform about one week before the conference. We also wrote a general guide for session hosts on how to set up all the parameters to make the sessions work the way we want it. This technical aspect of running a conference certainly requires some practice and we highly recommend thorough preparation.
3 Attendee Feedback Survey
We ran a feedback survey after the conference, which was answered by 114 participants (over 42% of the registered participants). Here, we present a selection of its results.
What is your current occupation? (114 responses)
Do you have a Ph.D.? (114 responses)
Which continent were you on during the conference? (114 responses)
Which plenary sessions did you attend? (98 responses)
How many ICDT-only sessions did you attend? (114 responses)
How many EDBT-only sessions did you attend? (114 responses)
Would you have attended more sessions if the conference would have been physical? (114 responses)
How did the online video presentations compare to conventional conference talks? (114 responses)
Notice that this is a rather even binomial distribution, slightly weighted to the negative.
What is the ideal length of a research talk for an online conference? (111 responses)
As an attendant, which kind of model(s) do you prefer? Multiple choices possible. (113 responses)
(1) Live talks, live Q&A, and having recorded talks available after the conference
(2) Streamed talks, live Q&A, and having the videos available after the conference
(3) Talk videos available beforehand and asynchronous Q&A (e.g. over Slack)
(4) Talk videos available beforehand and live Q&A in discussion sessions
Which meeting mode do you prefer for keynotes? (111 responses)
Which meeting mode do you prefer for research talks? (111 responses)
Was the software infrastructure adequate for supporting EDBT/ICDT? (114responses)
How many Slack channels have you joined during the whole conference? (109 responses)
Did the conference need more social interaction? (111 responses)
Did you attend the Bring Your Own Beer sessions? (111 responses)
Did you like the Bring Your Own Beer sessions? (30 responses)
In this question, we only selected the answers from respondents who said that they attended the Bring Your Own Beer sessions.
Do you have suggestions for more social interactions or networking?
This was a free-text field in the questionnaire. We received suggestions such as:
• Breakout rooms during coffee breaks, where you can see who is in which room.
• Dagstuhl-like breakout rooms (randomly put people together).
• Focused discussion rooms, e.g., per scientific area or per need ("looking for a phd student")
• Dedicated Ask Me Anything (AMA) with speakers (at least for keynotes).
• Playing some online game together.
• Imitate a "social event" like watching a live-streamed video/concert together.
Given the short timespan, our planning of EDBT/ICDT 2020 indeed focused mostly on simply making the scientific part of the conference work. There is much more that can be done in terms of social interaction in an online conference. Social interaction sessions could act as a buffer alongside the programme, i.e., they could be running before/after/during the scientific program to further accommodate time zone differences.
As a presenter of a research talk, would you prefer to present live or with a prerecorded talk? (80 responses)
As a presenter of a demo, would you prefer to present live or with a pre-recorded talk? (36 responses)
Among the suggestions for "Other", respondents wrote suggestions like giving live commentary on a pre-recorded presentation of the system, or having a short (pre-recorded?) presentation to give the audience the context and then a live demo.
Assume that EDBT/ICDT would be held physically. Would you attend virtually if this option existed? (113 responses)
For which reasons would you consider to attend a hybrid conference virtually? (107 responses)
Among the other reasons were: being able to attend more conferences per year, being able to join virtually even when you don't have a paper, being able to network more, being able to buy virtual registrations for internship students, no visa restrictions, less travelling.
Assume that EDBT/ICDT would be held virtually only. Would you attend?
Would you support the idea of having hybrid conferences for reducing CO2 emissions? (112 responses)
Would you support the idea of alternating physical meetings with purely virtual conferences for reducing CO2 emissions? (112 responses)
Was EDBT / ICDT 2020 better or worse than a typical other edition? (114 responses)
Was attending EDBT/ICDT 2020 better or worse than what you expected a virtual conference to be like? (110 responses)
Judging from these last two questions, attendants found that the online experience was indeed somewhat less than the physical experience, but at least the experience was better than what was expected from a virtual conference. Furthermore, we need to keep in mind that this edition was planned and organized in just three weeks, without any external guidelines. There is room for improvement.
4 Conclusions and Advice for Future Events
First-time organization of an online conference is a complicated matter, especially under tight time constraints. In our case, a great effort of coordination was needed and a task force (formed by the people co-authoring this report) made the executive decisions and carried out the re-quired work. On the other hand, once the executive decisions have been made, the organiza-tional amount of work is reasonable. We therefore encourage other conferences to try out the transition to an on-line mode in the short term. In the medium to long term, on-line and/or hybrid conferences may help the community reduce its CO2 footprint.
With a long-standing experience and within a large time window, things can be arranged more carefully. For instance, one could think about the following issues:
- Carefully choosing the underlying technological platforms on which the conference has to be hosted. People are aware of security issues around Zoom, but there is no available equivalent open-source tool that can host the same number of participants. Since a high number of participants is needed for plenary sessions, hopefully such open-source tools will be available in the long run. Dedicated platforms for scientific conferences are urgently needed in that respect.
- Several sessions that require tighter interactions, such as poster and demonstration sessions, need to be planned carefully. For instance, for demonstrations and posters, one would rely on the breakout rooms in Zoom to let people gather around a demo booth or a poster (with limited number of participants). If the poster or the demonstrated tool can be shared with the participants beforehand, the sessions can be also prepared in advance and be more fluent and interactive. Other sophisticated solutions, such as virtual reality and avatar-based video and chat tools, may be needed in the long run. These tools would help reproducing the physical interactions needed for poster and demo sessions along with the serendipity of meeting people with similar interests at these sessions.
- Networking would greatly benefit from having dedicated online sessions that are scheduled alongside the normal scientific sessions of the conference. Networking is truly the pitfall of an online event and this is especially deleterious for the junior members of our community. An idea would be to prepare networking well in advance and to pin interesting topics or discussions with colleagues of other universities and research teams (a sort of Pinterest specialized for scientific conferences).
Finally, we are pleased to share our experience at online EDBT/ICDT 2020 and eager to learn more about virtual scientific events in the near future. During the climate change session, which has been hosted by the conference this year, we had a lively and stimulating discus-sion about adopting CO2 plans for conferences. One of the options there is to allow alternate virtual and in-person events or hybrid (simultaneously virtual and in-person) events and thus contribute to reducing the environmental footprint of scientific conferences. Our online survey gives us two hopeful signs. First, there is a significant support of the community for going on-line in order to reduce CO2 footprint, and second, attendees clearly found this year's conference better than what they expected a virtual conference to be like.
Finally, in the spirit of moving open science and open access forward in the right direction, the videos of the conference talks are going to be made available to the general public. For EDBT, they can be linked directly from the proceedings and ICDT is looking into a similar solution.
Angela Bonifati is a professor in Computer Science at Lyon 1 University, France, and was program chair for EDBT 2020. Giovanna Guerrini is an associate professor in Computer Science at the University of Genova, Italy, and a member of the EDBT Executive Board. Carsten Lutz is professor in Computer Science at the University of Bremen, Germany, and was program chair of ICDT 2020. Wim Martens is professor in Computer Science at the University of Bayreuth, Germany, and is Council Chair of ICDT. Lara Mazilu is a Research Associate in Computer Science at The University of Manchester, United Kingdom, and was a volunteer for organising EDBT/ICDT 2020. Norman Paton is a professor of Computer Science at The University of Manchester, UK, and was President of the EDBT Association for EDBT/ICDT 2020. Marcos Antonio Vaz Salles is associate professor of Computer Science at the University of Copenhagen, Denmark, and was general co-chair for EDBT/ICDT 2020. Marc H. Scholl is a professor of Computer Science at the University of Konstanz, Germany, and a long-time member of the EDBT Executive Board. Yongluan Zhou is professor of Computer Science at the University of Copenhagen, Denmark, and was general co-chair for EDBT/ICDT 2020.
No entries found