Communications of the ACM
Contributed articles
The Dawn of Crowdfarms
Credit: DC Studio
Crowdsourcing is the process by which organizations or individuals outsource tasks with an online "open call."9 With tasks posted, and instructions and finished goods digitally exchanged, crowdsourcing enables the geographically distributed online workforce and work solicitors to cooperate on various tasks—improving productivity, social mobility, and the global economy.
Key Insights
Common crowdsourcing practice, illustrated by Amazon Mechanical Turk, comprises the completion of tasks by crowdworkers as opposed to solely computational systems. This approach has achieved impressive results in data clustering, content labeling, and other small tasks that individuals can complete in a short time. However, this has limited the opportunities for crowdworkers to collaborate and develop specialized skills while preventing crowdsourcing to be applied to projects that require higher levels of expertise and closer teamwork, such as software development and industrial design.
As crowdsourcing platforms and practices mature, will they reach a steady state and continue to grow while still focusing on simple tasks? Or, could there be a shift or disruption?
In previous research into Chinese crowdsourcing, we identified a new paradigm that could indicate a shift: small companies that regard crowd-work as part of their formal business and assemble teams to take on multi-faceted crowdsourced tasks requiring specialized expertise. We refer to these companies as "crowdfarms."a A crowdfarm is a small but growing crowdsourcing workforce in China that is positioned between traditional crowdsourcing and consultancies. A similar focus recently appeared in Up-work, which unveiled an "Agency Experience" policy to support small firms that specialize in complex, high-value crowdtasks.7 The emergence of these small businesses in both Eastern and Western crowdsourcing contexts indicates that organizational participation in crowdwork could become a widespread trend.
This article describes a series of studies conducted through interviews to obtain an in-depth understanding of this emerging organizational form. We describe how Chinese crowdfarms that were early adopters of this form operate, explore the perspectives of people who work in them, and assess the implications for the evolution of crowd-sourced work.
Crowdsourcing Platforms in China
By 2017, China's rapidly growing digital economy was estimated to include 30 million crowdworkers serving around 200,000 clients worldwide.15 Two of the largest Chinese platforms, ZBJ.com and EPWK.com, post millions of tasks. Initially, tasks were simple and self-contained, such as image labeling and data clustering. These appeal to Chinese crowdworkers who usually work part-time at home on tasks that are of personal interest and relatively short, requiring only a few hours to a few days to complete.
As processes were established and trust in the platforms grew, some customers posted more complex tasks that required a few weeks of work and a range of skills, such as software and game development, video and film production, and industrial and interior design. This increase in task complexity stimulated demand for specialization and collaboration in the Chinese crowdsourcing market. While some of these more complex tasks could be handled by a very skilled solo crowd-worker, small companies with teams engaged in similar work offline were also aware of these tasks.
Platform companies moved to support these high-revenue engagements with special bidding and management processes. ZBJ promotes itself as an "incubator," assisting more than 100,000 knowledge workers to grow and perhaps form or join companies. Since 2016, ZBJ has provided cheap rent and customized workspaces (which they call "ZBJ factories") to host crowdfarms that accept work posted on ZBJ, creating crowdsourcing communities in a hospitable business environment.
The crowdfarm work model enhances the role of crowdsourcing by tackling more advanced projects for requesters.
Government policies. Chinese government initiatives support the gig economy, including crowdfarms. The "mass entrepreneurship and mass innovation program"21 provides low taxation and subsidies for "Internet companies," including space, easy access to government services, and financial assistance. This and other enabling policies—together with a large, well-educated Chinese digital workforce; enthusiasm for working online; and the benefits of working for a registered company, such as health insurance and a pension—created a favorable landscape for this novel form of crowd-sourcing.16,18
Crowdfarms. Many crowdfarms are companies with five to 20 workers who engaged in offline business before crowdwork and now engage in both, with crowdsourcing becoming increasingly important. For some, it has become the primary source of income. In this process, a company and its employees expand their business acumen and normalize management processes to include more complex work arrangements involving managers and internal teams organized by expertise; recruitment that leads to labor contracts and social protections, such as pensions and insurance; and a reasonable wage system with a monthly basic salary and commissions based on personal contributions to both offline and crowdsourced tasks.
Crowdfarms are external providers of labor that differ from consultancies, outsourcing, and contractors. Crowd-farms do not typically provide strategic advice or reduce customer costs by taking over a business function. They do not procure work through email, telemarketing, online advertising, or offline marketing.5,17 By avoiding these expenses, they can be a lower-cost alternative for tasks that require a few days to a few weeks of work.
Our Crowdfarming Study
We encountered crowdfarming in 2019 while conducting surveys and interviews about crowdsourcing practices.18,20 After reporting on crowdfarms in a CSCW 2019 poster,19 we conducted interviews on several aspects of crowdfarming, which became the focus of this article. We sought a deeper understanding of the paradigm: how these businesses operate on a daily basis, how their employees feel about crowdwork, and how crowd-farms affect the overall crowdsourcing context. We recruited participants by posting a request for half-hour interviews as a task on ZBJ, for which they were paid 100 CNY (approximately 14 USD). We recruited crowdfarm workers and solo workers.
Crowdfarm workers were told that to qualify, they should work for companies that take on crowdsourced tasks as part of their formal business. To protect their privacy, they were instructed to use ZBJ's chat system to indicate their intention to participate and provide a phone number. Participants were also informed that all data would be anonymized and that they were welcome to contact researchers with any concerns.
Twenty-three crowdfarm workers contacted us, each from a different company. After an initial online discussion to confirm that they worked in crowdfarms and to address concerns, such as assuring them that their company name would also be anonymized, we scheduled interviews with all of them. Most interviews lasted between 30 and 60 minutes and all were conducted in Chinese.
What follows are our high-level findings from the interviews, supplemented with additional research. We discuss:
- The procurement and carrying out of tasks by crowdfarms and reputation management following task completion.
- The work experience of crowdfarm workers, including their perceptions of the work settings, problems encountered while completing tasks, and rewards and work/life balance.
- The potential impact of crowd-farms on solo crowdworkers and task requesters, as well as broader implications this paradigm carries for crowd-sourcing platforms.
Work Practices of Crowdfarms. Procurement of tasks. Solo crowdworkers look at a posted task and decide whether they have the skills to complete it on time. Sometimes they attempt tasks beyond their current abilities as a way to develop their knowledge and skills. In crowdfarms, more often than not it is managers who make the selection. They may prefer large and complex tasks that offer higher payment, but those we interviewed focus on task feasibility: Is the request in a field they specialize in? Do they have the expertise to complete it on time?
Our participants attributed this relatively cautious crowdfarm strategy to a couple of considerations: The financial loss from a failed larger and more complex task versus that of a solo crowdworker who walks away from a short task, and the loss of further business from a failed task in contrast to the potential for additional business and positive referrals from one that succeeds.
After careful task selection, crowd-farms bid on the tasks via a mechanism designed by Chinese crowd-sourcing platforms for large and complex tasks. A crowdfarm publicly posts an intention to participate on the task page. The crowdfarm may directly contact a requester in private to clarify the requirements and negotiate payment terms. Because the payment for large tasks is relatively substantial, the bidding mechanism requires both parties to sign legal contracts after confirming their intention to collaborate. Only after the requester deposits part of the payment on the platform as a guarantee does a crowdfarm begin the work. Although the bidding mechanism is designed to facilitate and nurture collaboration between crowdfarms and requesters, private direct contact decreases the transparency of the process. Participants confirmed reports3,4 that a lack of transparency in crowdsourcing can adversely affect trust, satisfaction, and motivation.
Carrying out tasks. Crowdfarms usually carry out a procured task by managing it internally. A manager could allocate a task that is difficult to decompose, such as logo design, directly to a specialized crowdfarm worker. However, decomposable tasks are divided into smaller work units and assigned to internal team members based on expertise or suitability. The manager then supervises work progress to ensure it is on schedule. After the crowdfarm teams or individual workers finish their tasks, they collaborate in their shared workplace to integrate the parts and submit the deliverable to the requester for feedback. If solo crowdworkers collaborate, they generally do so remotely, but crowd-farm face-to-face collaboration is seen as improving worker productivity, especially for urgent tasks.1 One participant described how his IT crowdfarm operates on a daily basis (all quotes are translated from Chinese):
"I usually start working at 9:00 A.M. The first thing I do is check if there are tasks that we can undertake on the ZBJ platform, and then I will spend two to three hours communicating with requesters in terms of the requirements of the tasks we already procured… In general, I divide these tasks into small parts and assign them to workers based on their expertise. I, myself, usually program the functions, and we also have workmates who are good at PHP language or specialized in software documentation … We usually have a meeting every one or two days to discuss the problems encountered in doing tasks and the future work plan … Our company uses a software system to allocate tasks and supervise the work progress of workers so that we can easily integrate and test all work units before we submit to requesters." (P1, 50-year-old male.)
When a crowdfarm cannot handle part of a task, it subcontracts the work to external business partners, solo crowdworkers, or other crowdfarms. The crowdfarm might select a company known to possess the necessary skill set, or, like a typical requester, crowdsource the work unit with relatively low remuneration to maximize profit. After paying secondary workers, crowdfarms integrate the work and submit it to their requester. Subcontracting by crowdfarms is not unusual: Collaboration among companies is a common strategy for small Chinese enterprises with insufficient means to support an end-to-end business.13 Like crowdfarming itself, subcontracting is a short-term arrangement and not outsourcing a business activity.11 To control the quality of subcontracted tasks, crowdfarms usually ask external contractors to regularly update work progress and send reports/prototypes to requesters for feedback. However, the crowdfarming companies had mixed feelings about subcontracting transparency. About half believed that it is a requester's right to know how their tasks are handled while the other half worried that subcontracting may result in requesters questioning their professional capabilities and can also negatively impact the possibility of future business.
Reputation management. Solo crowdworkers rarely interact with requesters before receiving a final direct acceptance or rejection of their submission. They manage their reputation by proactive, preventative tactics, such as sticking to familiar tasks, pre-task training, and returning a task as soon as it is found to be too difficult.6 In contrast, crowdfarms that take on larger, complex, and lengthy tasks receive oversight and feedback from requesters over days or weeks, enabling them to preempt difficulties and fix issues that could lead to rejection. After a project has closed, a crowdfarm uses post-task and compensatory strategies to manage its reputation. The crowdfarm asks requesters for positive ratings and comments that can be displayed in the crowdfarm's profile, carries out extra and usually unpaid work to refine submitted tasks, and offers price discounts if necessary to obtain more favorable final feedback.
One manager/worker in a design company explained how he obtained a favorable rating from a tough requester:
"There was a requester who insisted on giving us negative feedback, as one of my staff quarreled with him over excessive requirements. So I apologized to him and offered him a few new designs for free, but it did not work out… In the end, though we had already signed the contract, I had to give him a discount on the price in order to get him to give us positive feedback." (P2, 31-year-old male)
The Work Experience of Crowdfarm Workers. Perceived work-setting advantages. Crowdfarm workers we spoke with report being generally satisfied with their work conditions. They appreciate that managers can provide timely and effective help when they encounter difficulties, which enables them to handle tasks more efficiently than they would on their own. They also appreciate that crowdfarms "feed" them remunerative tasks that match their professional skills, and provide financial and social protection in the form of a monthly wage and health insurance—benefits not available to solo crowdworkers.
Perceived problems in crowdwork. The most frequent complaint in interviews was that remuneration for tasks was decreasing. Although payment varies across task type and domain, price competition among crowdfarms is driving down payment for tasks, affecting the bonuses that crowdfarm workers earn.19 Relentless bidding competition could create a race to the bottom in the Chinese crowdsourcing market. Another major problem reported by crowdfarm workers was ill-defined requester requirements. Solo crowd-workers who undertake short tasks are given relatively straightforward requirements, but crowdfarm workers must resolve complex requirements in the ongoing process we described. An unclear or ambiguous requirement can increase the cost of communicating with requesters and affect the timing and final quality of the completed work, impacting the crowdfarm worker's remuneration.
One crowdfarm worker identified concerns and frustrations also expressed by others:
"I took a task that looked like they needed a simple online platform with a 100,000 CNY budget (approximately 14,490 USD). However, it turned out that the requester had no idea what they wanted…In the end, we received only 1/10 of the money as they thought we did not meet their ever-changing and unrealistic requirements, though we had already provided the general framework of the website…I know a [crowdfarm] on ZBJ in our city. Their strategy is to bid with extremely low prices to win the tasks. To be honest, if it was not because the platform gives them more opportunities to attract customers with low prices, I seriously doubt these workers could support themselves with such low profits in tasks." (P3, 30-year-old male)
Rewards from crowdwork. Although many crowdfarm workers also earn money from offline businesses that their companies procure locally, they use crowdwork income for basic living expenses or to improve their quality of life, such as to support hobbies. In addition to the monetary reward, "guanxi"—a Chinese concept of an interpersonal relationship involving obligation, commitment, and exchange of favors—with requesters is generally regarded as a very important nonmonetary reward. Given that Chinese business transactions often result from a successful guanxi,2 working for a well-connected requester can reduce potential problems at work (for example, post-task refinement), enabling tasks to be completed in a timely and smooth manner. In fact, for some of our participants, guanxi with requesters is of greater significance than the remuneration. Therefore, in contrast to the smaller tasks involving little or no communication, frequent interactions with requesters in complex crowdwork can obviously shift worker focus from direct task payment to work relations. Business collaborations with requesters in good guanxi bring workers more stable task payments and richer work experiences that assist personal career development.
A crowdfarm worker who specializes in visual identity reported:
"The most important reward we gained from crowdsourcing is that we have reached more customers and established business relationships with them…Although we usually cannot make much money from the first task we do for one, after gaining their trust, they will contact us with more tasks. Then the profit for a task will be much higher." (P4, 49-year-old male)
Work-life balance of crowdfarm workers. In contrast to Chinese solo crowdworkers, who rarely undertake tasks when sick and rarely repeatedly work overtime,20 crowdfarm workers report high levels of stress and exhaustion, with less time spent on leisure and family activities. As full-time employees with tasks mandated by managers, crowdfarm workers cannot flexibly control task volume or scheduling. This is influenced by the 9-9-6 work culture of many Chinese IT companies: 9:00 A.M. to 9:00 P.M., six days a week.12 The crowdfarms in our study are small Internet companies that often rely on employees working overtime to stay in business. To prevent further disruption to their lives, crowdfarm workers report talking less about their work with families and friends, creating a clear boundary between work time and family time.
Discussion
Crowdsourcing is generally regarded as an emergent work paradigm with a disruptive business model that deviates from traditional business operations. Could crowdsourcing itself be disrupted? We found that traditional management approaches play a significant role as crowdsourcing scales and is applied to tasks requiring closer internal collaboration and coordination.
Crowdfarms share the online task crowdwork platform category with Upwork and TaskRabbit, in contrast to crowdwork platforms offering asset-based services, such as Airbnb or Uber, "Playbour" ("playful labour" platforms), and profession-based freelance platforms such as iStock-photo.8 They too mix typical solo crowdworkers and small businesses and projects ranging from simple to complex, so the emergence of crowd-farms on ZBJ could provide these platforms with an intriguing alternative model to organize and incubate their crowd workforce.
The crowdfarm work model enhances the role of crowdsourcing by tackling more advanced projects for requesters. It accelerates the accomplishment of open-ended and complex tasks that cannot be easily decomposed into the smaller tasks posted on microtasking platforms, such as Amazon Mechanical Turk. The cost of creating the workflow to address a complex task can be high, even for large corporations. Crowdfarms, matching their skill sets to specific complex task requirements, can produce workflows more cost-effectively than in-house workflow development. By scaling up the number of complex tasks that can be carried out efficiently, these companies accelerate complex work production, which is highly valued in knowledge and innovation-seeking economies.
Crowdfarms are impacting the vast solo crowdworker community, the individuals and enterprises that request crowdwork, and the platform companies. For solo crowdworkers, crowdfarms are a double-edged sword. By breaking down procured tasks into smaller work units and subcontracting some of them, they add low-complexity tasks to the crowdsourcing market. On the other hand, profit-oriented crowdfarms use their advantages in teamwork and professionalism to procure as many tasks as possible, leaving solo crowdworkers with specialized skills increasingly competing at a disadvantage with specialized companies. Crowdworkers who subcontract or join a crowdfarm must share income from their work with crowdfarm management. They obtain employment benefits, but are also prone to work-life-balance stress.
Crowdfarm face-to-face collaboration is seen as improving worker productivity, especially for urgent tasks.
Crowdfarms and the larger crowd-sourcing field can mix in different ways. Solo crowdworkers could join a crowdfarm. Crowdfarm employees who have developed specialized skills could become independent freelance contractors or form their own companies.10
Requesters take on some risks but obtain clear benefits: an efficient, professional, one-stop crowdsourcing approach to enlist required expertise. Requesters do not have to break down complex tasks and integrate the resulting work. Nor do they need to find and communicate with multiple competent solo crowdworkers. The risk for requesters, especially given that crowdfarms sometimes subcontract tasks without informing them, is uncertainty about who is performing a task and the quality of the final submission. This risk is offset by the contracts managed by the platforms, quality control within a crowdfarm, and the strong desire of crowdfarms to establish guanxi and earn positive recommendations. One experimental study found that in some situations, subcontracting by a crowdworker led to superior outcomes.14
Platform companies responded to the opportunity to take on more lucrative tasks by offering space to crowdfarms and by mediating high-stakes crowdsourcing tasks for requesters and crowdfarms. Our interviews identified points of friction that platform companies are likely to address: a need for better tools for requester-crowdfarm communication and better strategies for regulating and normalizing subcontracting to protect subcontractors. Platform companies monitor for fake requests22 and could also detect problematic bids that are significantly lower than payments for similar tasks. Imbalances in remuneration could affect crowdfarms and their employees, as well as solo crowdworkers, and undermine the crowdsourcing industry.
Our study of crowdfarms provides an initial insight into an emerging and evolving crowdsourcing phenomenon in China. With its significant differences from solo-worker-based crowdwork, as well as its impacts on the general crowdsourcing context, crowdfarming can help shape our conception of the future of work.
Although payment varies across task type and domain, price competition among crowdfarms is driving down payment for tasks.
We do not yet have a complete picture of crowdfarming. We hope our work motivates others to join in research and discussion around this phenomenon. It is not clear how crowdfarms, rooted in a Chinese crowdsourcing community that is 30-million strong and supported by platforms and the government, will evolve, or how the experience of crowdfarm workers will change. Moreover, considering that the U.S.-based platform Upwork is now encouraging small firms to participate in crowdwork, it is not known how ZBJ's crowdfarms and Upwork's boutiques differ in terms of technology, policy, legal framework, and labor characteristics. These differences could have a profound effect on the evolution of the gig economy and the well-being of millions of people now engaged in crowdsourced tasks.
References
1. Battiston, D., Blanes i Vidal, J., and Kirchmaier, T. Face-to-face communication in organisations. SSRN Electronic Journal (2017), doi:10.2139/ssrn.2934290.
2. Buckley, P.J., Clegg, J., and Tan, H. Cultural awareness in knowledge transfer to China—The role of guanxi and mianzi. J. of World Business 41, 3 (2006), 275–288. doi:10.1016/j.jwb.2006.01.008.
3. d'Eon, G., Goh, J., Larson, K., and Law, E. Paying crowd workers for collaborative work. In Proceedings of the ACM on Human-Computer Interaction 3 (2019), 1–24, doi:10.1145/3359227.
4. Fieseler, C., Bucher, E., and Hoffmann, C.P. Unfairness by design? The perceived fairness of digital labor on crowdworking platforms. J. of Business Ethics 156, 4 (2019), 987–1005, doi:10.1007/s10551-017-3607-2.
5. Gummesson, E. The marketing of professional services—An organizational dilemma. European J. of Marketing 13, 5 (1979), 308–318, doi:10.1108/EUM0000000004951.
6. Gupta, N., Martin, D., Hanrahan, B.V., and O'Neill, J. Turk-Life in India. In Proceedings of the 18th Intern. Conf. on Supporting Group Work, ACM Press (2014), 1–11, doi:10.1145/2660398.2660403.
7. Hippler, K. From freelancer to agency: Is it the right move for you? Upwork (May 3, 2020), https://www.upwork.com/resources/from-freelancer-to-agency.
8. Howcroft, D. and Bergvall-Kåreborn, B. A typology of crowdwork platforms. Work, Employment, and Society 33, 1 (2019), 21–38, doi:10.1177/0950017018760136.
9. Howe, J. The rise of crowdsourcing. Wired (2006), https://www.wired.com/2006/06/crowds/.
10. Kauser, H. Can crowdsourcing steal the thunder from traditional consulting? Beroe (2017), https://www.beroeinc.com/article/crowdsourcing-consulting/.
11. Kavčič, K., Markič, M., Mesko, M., and Ježovnik, A. Strategic management of outsourcing. Faculty of Management (2014), http://www.fm-kp.si/zalozba/ISBN/978-961-266-159-5.pdf.
12. Li, Q. and Zhong, R. '996' is China's version of hustle culture. Tech workers are sick of it. The New York Times (2019), https://www.nytimes.com/2019/04/29/technology/china-996-jack-ma.html.
13. Liu, X. SME development in China: A policy perspective on SME industrial clustering. SME in Asia and Globalization 5 (2008), 37–68.
14. Morris, M.R., Bigham, J.P., and Brewer, R. et al. Subcontracting microwork. In Proceedings of the 2017 CHI Conf. on Human Factors in Computing Systems, ACM Press, 1867–1876, doi:10.1145/3025453.3025687.
15. Shengping, H., Chijian, Z., and Tu, H. On the research of intelligent coupling and coordination between employers and Witkey on the Crowd Innovation Network Platform. J. of Xiangtan University 41, 1 (2017), 94–101, doi:0.13715/j.cnki.jxupss.2017.01.017.
16. To, W-M. and Lai, L.S.L. Crowdsourcing in China: Opportunities and concerns. IT Professional 17, 3 (2015), 53–59, doi:10.1109/MITP.2015.47.
17. Vigato, V and Markova, E. Transition to consulting: Marketing & sales activities & tools. LAB University of Applied Sciences, Lahti, Finland, (Dec. 31, 2016).
18. Wang, Y., Papangelis, K., and Lykourentzou, I. et al. In their shoes: A structured analysis of job demands, resources, work experiences, and platform commitment of crowdworkers in China. In Proceedings of the ACM on Human-Computer Interaction 4 (January 2020), 07:1–07:40, doi:10.1145/3375187.
19. Wang, Y., Papangelis, K., Lykourentzou, I., and Khan, V-J. The changing landscape of crowdsourcing in China: From individual crowdworkers to crowdfarms. 2019 Computer Supported Cooperative Work and Social Computing Conf. Companion, ACM Press, 413–417, doi:10.1145/3311957.3359469
20. Wang, Y. et al. Crowdsourcing in China: Exploring the work experiences of solo crowdworkers and crowdfarm workers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, (2020), 1–13, doi:10.1145/3313831.3376473.
21. Yongwei, Z. and Junkuo, Z. Mass entrepreneurship and innovation: New impetus to development. DRC of the State Council, http://www.chinadaily.com.cn/m/drc/2016-04/07/content_24350321.htm.
22. Zhu, F., Chen, W., and Sun, S. ZBJ: Building a global outsourcing platform for knowledge workers (A). Harvard Business School (January 26, 2018), https://www.hbs.edu/faculty/Pages/item.aspx?num=53948.
Footnotes
a. Our use of "crowdfarm" employs the English concept of "farming work out" (that is, outsourcing) to identify this as a unique kind of firm, different from crowdsourcing platforms or firms that post crowdsourced jobs. Chinese IT workers sometimes self-mockingly refer to themselves as "ma nong" ("code farmers" in English) to describe the heavy pressures they encounter in digital work, and to support their identity and foster camaraderie with those doing similar jobs. Given the mixed connotations of the term "ma nong" or "code farmer," we use the respectful "crowdfarm worker," not "crowdfarmer," to avoid any appearance of insensitivity toward people whose work we value.
©2022 ACM 0001-0782/22/8
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2022 ACM, Inc.
No entries found