Communications of the ACM
News
Following the Crowd
Ushahidi enables volunteers to map everything from natural disasters to political turmoil.
Credit: Erik Hersman
If one thing is entirely clear about the Internet it's that today's ability to democratize information and tasks is nothing short of remarkable. Increasingly, groups aggregate knowledge through wikis, track incidents during a political uprising or emergency through text messages and email, and create instant teams and organizations in order to solve tasks and accomplish work.
"There's an ability to mobilize information and groups quickly and effectively," observes Peter Lee, former director of the Transformational Convergence Technology Office at the U.S. Defense Advanced Research Project Agency (DARPA) and currently director of research at Microsoft. "This capability is fundamentally changing societyand the way we approach common tasks and problems."
At the heart of this equation is crowdsourcing. The concept revolves around large groups of people or a community handling tasks that have traditionally been associated with a specialist or small group of experts. Jeff Howe, author of Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, coined the term in 2006, citing technology as a way to draw a greater number of people into large tasks while tapping knowledge and expertise that previously flew under the radar.
Over the last few years, crowdsourcing has emerged as a viable solution for businesses, relief agencies, researchers, politicians, the military, and others looking to grab bits and bytes of information in a nontraditional and decidedly more chaotic way. Howe describes it as a way for many to do the work and tasks previously handled by a few. Crowdsourcing has social, economic, cultural, business, and political implications, he says.
Crowdsourcing is gaining momentum across a wide swath of industries and organizations. However, the concept isn't without controversy. Some organizations have found that crowd-sourcing is expensive and unreliable. Even those that have used it successfully have found that bad data and faulty observations sometimes get tossed into the mix. The old axiom "you get what you pay for" can become glaringly apparent when anyone can join the fray.
Groupthink Unleashed
Crowdsourcing is based on a simple but powerful concept: Virtually anyone has the potential to plug in valuable information. As Howe noted in Crowd-sourcing, "Technological advances in everything from product design software to digital video cameras are breaking down the cost barriers that once separated amateurs from professionals. Hobbyists, part-timers, and dabblers suddenly have a market for their efforts [as organizations] tap the latent talent of the crowd."
The roots of crowdsourcing extend back to the 1990s. That's when individuals and institutions began volunteering spare computing cycles to help solve major research projects involving everything from mathematical formulas to medical problems. This community-based approach extended to wikis and other collaboration tools in the age of the Internet. In today's Web 2.0 world, peer-to-peer and collaboration-based platforms play an increasingly important role in an array of fields.
It's safe to say that crowdsourcing is a thoroughly disruptive tool. "Normally," Lee says, "business, science, and high-tech development takes place in fancy laboratories or in academic ivory towers. The idea of taking the development process out to the public is alluring yet intimidating. When such a powerful technology is unleashed it leads to unpredictable and sometimes surprising results."
Many organizations are now developing their own crowdsourcing software or platforms. These systems typically tap into mashups and other Web 2.0 tools accessible through a standard Web browser. In some cases, these applications rely on mapping software such as Google Maps. Others depend on wiki-type software to collect observations, comments, and other pertinent data. Along the way, a person or software application populates a data-base or a mashup to provide actionable information.
One of the most successful commercial crowdsourcing platforms is Amazon's Mechanical Turk. It offers businesses and developers access to an on-demand, scalable work force. Essentially, potential employers post tasks and workers select jobs they would like to perform. Payment is established up front and fund transfers take place through Amazon.com.
In fact, an increasing number of organizations are turning to crowd-sourcing to reengineer an array of processes. The military is exploring ways to collect intelligence data through crowdsourcing, government agencies are using it to collect data on everything from road repairs to urban planning, and relief agencies are turning to it to better understand how to focus aid and resources.
One organization that's leading the way with crowdsourcing is Ushahidi. The Web site's software platformcollaboratively written by developers in Ghana, Kenya, Malawi, Netherlands, South Africa, and the U.S.enables volunteers throughout the world to map everything from natural disasters to political turmoil. The open source platform, now used by dozens of organizations, was developed to track reports of violence in Kenya in the aftermath of its disputed 2007 presidential election.
Ushahidi is based on a simple enough concept: as an event unfolds, a volunteer on the ground sends a brief report through a Web browser or text message and software maps the entry by time and location. Organizations can download the free software and deploy it as they see fit. Ushahidi (which means "testimony" in Swahili) incorporates powerful content management capabilities, a robust database, and server-side map clustering. When data from media, nongovernment organizations, and citizens is tossed together, the result is a mashup that provides powerful geographical mapping tools.
Over the last few years, Ushahidi has been used by organizations to pinpoint medical needs associated with an earthquake in Haiti; monitor local elections in Afghanistan, India, and Mexico; map incidences of violence in Pakistan; track medicine shortages in the Philippines; and analyze human rights abuse in the Congo. Its uses are limited only by the creativity and needs of those using it. "It produces real-world results," says Patrick Meier, director of crisis mapping for Ushahidi.
In today's Web 2.0 world, peer-to-peer and collaboration-based platforms play an increasingly important role in an array of fields.
Another organization that has tapped into the power of crowdsourcing is DARPA, the research and development office for the U.S. Department of Defense. "Computer technology has led to all sorts of surprising and disruptive outcomes," Lee explains. "Crowd-sourcing offers tremendous potential."
DARPA has experimented with a number of crowdsourcing initiatives, including the DARPA Network Challenge, a balloon hunt that involved more than 4,000 teams attempting to locate 10 moored, 8-foot tall, red weather balloons at 10 fixed locations across the continental U.S. (For more about the DARPA Network Challenge, see "Mechanism Design Meets Computer Science," August 2010.) The agency created incentives for participants to lie, keep secrets, and infiltrate each other's teams. "We wanted to simulate all the dirty tricks that would take place in a wartime military environment," Lee says.
The project encompassed the Internet, social networking, real-time communications, wide-area collaboration, and other crowdsourcing techniques. The winner, a team from Massachusetts Institute of Technology, received a $40,000 prize. It took the participants nine hours to complete the task. The team used a multi-level marketing strategy to recruit studentspaying as much as $2,000 for information about balloon coordinates and lesser sums to those who invited them (and the person who invited the inviter). DARPA studied the resulting social interaction and is using the data to formulate more advanced wartime strategies.
Advantages and Disadvantages
"Crowdsourcing offers both advantages and disadvantages," Ushahidi's Meier points out. "It is very efficientwith the right community in placeat gathering information quickly and effectively. It can help speed response and cut through the confusion that occurs during the initial stage of a disaster. It can quickly fill the information gap." What's more, he says, traditional surveys and techniques require more time and expenseoften with less impressive results.
Meier says that concerns about the accuracy of data aren't unfounded. "One of the challenges is developing trusted sources," he says. Of course, it's not possible to vet everyone. And restricting who posts data online defeats the entire purpose of crowdsourcing. In addition, some errors and inaccuracies are inevitable, even from well-meaning participants. "You have to operate under the assumption that most people are honest and most information is accurate," he adds. "But it's necessary to build in a margin for error."
Another challenge is publicizing a crowdsourcing platform and establishing a network of volunteers. It's a task that requires significant money, time, and effortsomething that many nongovernmental organizations lack. Lee says that organizations typically publicize efforts any way they canthrough press releases, a Web site, and word of mouth. However, higher participation rates translate into a greater volume of data, but sorting through it to spot what's relevant and useful can prove taxing. "Managing the process can be difficult," he admits.
Over the last few years, Ushahidi has been used by organizations to monitor local elections in India, map incidences of violence in Pakistan, and track medicine shortages in the Philippines.
Nevertheless, crowdsourcing continues to advanceand involve increasingly complex issues. In some cases researchers and computer scientists are attempting to attack age-old questions and challenges in new waysand gain fresh perspectives. For example, when Vinay Deolalikar, a renowned computer scientist at Hewlett-Packard labs, sent an email to top researchers claiming that P doesn't equal NP, it generated considerable interest.
But then, once the issue hit the blog of Richard J. Lipton, a computational complexity expert at the Georgia Institute of Technology, interest among other researchers and a lay audience peaked. An informal peer review process followed. Participants discovered errors and the level of interaction and exchange exceeded that of any traditional process. In the end, one researcher described the entire episode as a "Nerd Super Bowl."
Clearly, crowdsourcing is here to stay. "It is changing the way government, corporations, and others tackle complex issues and problems," Lee notes. "It is leading to an entirely different mindset about how product development, problem solving, and decision making take place."
Further Reading
Brabham, D.C.
Crowdsourcing as a model for problem solving: an introduction and cases, Convergence: The International Journal of Research into New Media Technologies 14, 1, Feb. 2008.
Brabham, D.C.
Crowdsourcing the public participation process for planning projects, Planning Theory 8, 3, August 2009.
Howe, J.
Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business, Random House, New York, 2008.
Lakhani, K.R., Jeppesen, L.B., Lohse, J.A, and Panetta, P.A.
The value of openness in scientific problem solving, Harvard Business School Working Knowledge, October 2006.
Leimeister, J.M., Huber, M., Bretschneider, U., and Krcmar, H.
Leveraging crowdsourcing: activation-supporting components for IT-based ideas competition, Journal of Management Information Systems 26, 1, Summer 2009.
Figures
Figure. Ushahidi enables volunteers to map everything from natural disasters to political turmoil.
©2011 ACM 0001-0782/11/0200 $10.00
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.
No entries found