Communications of the ACM
Communications of the ACM
Targeting Audiences on the Internet
Much useful research can be conducted with the help of an Internet audience, ranging from psychological experiments to market research surveys [7, 9]. Internet surveys are cheaper and faster to carry out than postal surveys, and their electronic format facilitates analysis. The Internet can also provide anonymity not available elsewhere, which helps reduce or eliminate the social desirability effect, whereby respondents give answers they think the surveyor expects [5]. Computer presentation yields more frank and complete answers to sensitive questions, according to the Computer-Assisted Personal Interviewing literature [12]. Research results via the Internet can be collected around the clock from individuals who may be difficult to contact or unwilling to participate using traditional research methods. The novelty of computer-aided research results in higher respondent interest and higher response rates.
We explored how survey respondents may be targeted on the Internet using a survey on mobile phone designs. To attract respondents, we advertised the survey in newsgroups and placed banner ads on several Web pages. The survey was also included in the databases of three Internet search engines. Email may also be used to target respondents. Broad-based, unsolicited spamming is illegal in some U.S. states, but using email to contact a highly selected, interested audience, to survey existing users of a software package for example, has received less criticism. Because we lacked a mailing list of individuals interested in mobile phones, we could not use this approach.
Our survey was directed toward three consumer groups: women, UK residents, and existing mobile phone users. Women were chosen as a target since they have traditionally been underrepresented online. UK-based Internet users are another relatively small but important group for our work. Mobile phone users were the specific user group. At the time this survey took place, Internet penetration levels in the UK may have been as low as 2% of households and 5% of businesses [1]. We used 10 versions of the survey, each with a different URL. Six versions were used for the newsgroups, three for the banner ads, and one for the search engines. Individuals were unlikely to see the survey in multiple places; no one responded to multiple surveys.
We targeted respondents on several Web sites using a banner-advertised incentive: "WIN a Mobile Phone: Complete a short survey and WIN a high- quality digital mobile phone." Women were targeted on a romance novel Web site called "Your Weekly Kiss." The ad was on the site's home page for 14 days, receiving an estimated 800 viewings. Mobile phone users were targeted on a mobile phone Web site called GSMag International. The banner was placed on the site's home page until it received 5,000 viewings or the survey was concluded. UK respondents were targeted with a 30-day banner ad on the UK Hyperbanner link exchange, a system that operates on the basis of reciprocal advertising. The incentive was also used to attract respondents on the Alta Vista, Infoseek, and Lycos search engines, as well as half of the newsgroups we targeted. (See Figure 1 for details of survey announcements.)
Incentives can be useful for conducting Internet surveys, and many see an exchange culture emerging, because Internet users may not be willing to give personal information without getting something in return [4]. When Georgia Tech introduced cash incentives in its sixth survey, it found the overall number of respondents did not increase significantly, but respondents provided more data [9]. After asking respondents to classify themselves as current users, potential buyers, or nonusers not likely to purchase a mobile phone, our survey explained that only fully completed surveys would be eligible for the prize incentive. When respondents submitted their survey responses, they reached a page that thanked them.
Survey Responses
Just over 1,000 individuals responded to our survey. Most responses came from search engines (750), followed by banner ads (225), and newsgroups (39). Figure 2 shows the response rate over the first 30 days. Responses surged over a 24-hour period that probably coincided with when the survey was listed on the search engines. Researchers at Georgia Tech have noted that surges in responses occur when surveys are announced on highly visible pages [8]. Respondents came from 49 countries, with the vast majority from North America, followed by Western Europe, Asia, and Eastern Europe. The distribution of responses mirrored the distribution of known Internet hosts [10], with a 0.98 correlation between responses and hosts by country. These results suggest the survey was widely distributed, reaching a broad cross section of the Internet, and that any country with Web sites, Web search engines, or dedicated newsgroups can be reached. The response rate to newsgroup postings was disappointing, being far less than the 2% often found with more traditional methods or banner ads [4]. The ratio of incentive to no-incentive responses was 3:1, and the group most likely to respond to the incentive, the mobile phone newsgroup, had a slightly higher response rate than the others. Table 1 shows the distribution of the newsgroup responses. The postings (incentive and non-incentive) were successful in targeting selected groups: postings on UK-resident newsgroups attracted responses only from the UK, postings on women's newsgroups yielded only female respondents, and postings on mobile phone newsgroups attracted responses from mobile phone users as well as potential buyers.
Computing the response rate for newsgroups is difficult because the number of postings probably underestimates the number of readers. Table 2 shows the three ways we computed newsgroup readership: using a previous survey of newsgroups by Brian Reid, of Digital's Network Systems Lab in Palo Alto [11], estimates from the Usenet Information Center (UIC) (sunsite. unc.edu/usenet-i/), and local newsgroup reading in July 1997 at the University of Nottingham's computer service.
UIC's readership numbers differ slightly from Reid's. (Reid estimated newsgroup readership statistics as a community service until July 1995.) UIC's numbers appear to be from early 1995, and may originate from an earlier, unpublished, readership survey, or from earlier versions of Reid's survey. Still, the three pictures of readership are relatively consistent. The readership estimates, if accurate, indicate these newsgroups represent relatively large populations of 20,000 to 200,000 readers.
Where numbers are not available we have entered estimates based on the response rate of its paired group. The estimates are useful for comparing the number of responses between groups. They suggest, for example, the response rate from the UK target newsgroups was higher than the female target newsgroups. We have also added an estimate of the readership of the alt.cellular.gsm newsgroup unadjusted for the incentive. If there had been no incentive and the readers of this group replied at the rate of the alt.dcom.telecom newsgroup, we would expect the readership to be 313,200. This would be surprising given the size of other newsgroups.
The response rate of banner advertising on Web sites unrelated to mobile phones was low, less than 1%. Banner ads on the mobile phone site yielded significantly more responses, almost 5%. These rates can be compared with previously reported rates of 12% [3]. Most responses were from the targeted groups (See Table 3).
Respondents must be carefully, appropriately, and respectfully targeteda responsible approach that also happens to produce better results.
The greatest number of responses, 750, came from individuals who found the survey through a search engine. Table 4 shows the distribution by target category. The self-reported demographics of this group were comparable with the other targeting methods with a few exceptions. These respondents were more likely to be female than other respondents. They tended to be older; more than 40% were over the age of 40. (This age group represented less than 10% of responses from banner ads and newsgroups.) And they were more likely to be from North America. Only 1% of people arriving at the survey site from search engines were UK residents, whereas for those arriving via newsgroups and Web sites, the percentages of UK residents were 18% and 10.5%, respectively.
Implications
The Internet provides an excellent opportunity to reach respondents quickly and cheaply, but senders should keep its limitations in mind. Based on this case study, we can suggest good practices we have found to work that should reduce the amount of effort for those sending and for those receiving numerous kinds of messages on the Internet.
Posting on newsgroups may be a good method to recruit participants for online focus groups, where large numbers are not required, but the characteristics of respondents are importantif you wish to recruit participants on the basis of nationality, occupation, or hobbies, for example. General requests are inappropriate and unwanted. The cost to readers as a group, and the loss of goodwill from those who take offense at the request, is quite high. Reputable firms will keep in mind the recipient must often pay to receive what to them is junk mail. Even when our incentive was mentioned, the survey received replies with a maximum of 1 per 5,000 of the theoretical exposures, with 1 per 10,000 more typical. Also, we received three responses noting our posting was inappropriate, equivalent to about 8% of the newsgroup responses. All the negative responses came from uk.* newsgroups, suggesting UK Internet users may be more sensitive to rules being breached, or feel they have more influence over a post from a UK address. Because of the relatively low response rate and these potential difficulties, we do not generally recommend using newsgroups to solicit respondents.
Banner ads and search engines present fewer difficulties. Using banner ads on appropriate sites can successfully target survey respondents. It is important to check where banners will be placed and to determine whether the site audience fulfills the respondent profile. To attract respondents with specific characteristics, one should mention an incentive appealing to them directly. Including a site and its keywords in search engine databases appears to be a fruitful and inexpensive method to target respondents, particularly North Americans. Targeting via search engines holds potential to generate a large response from a desired target group because respondents find the survey as part of their own search. Indeed, search engines are already using search topics to target their own banner ads.
The main drawback of targeting survey respondents via the Internet is the anonymity of respondents. Some users, mainly in chat rooms, are known to create alternate personae [12]. Multiple submissions can also be a problem as they invalidate data, and on the Internet they may be easier to create. Georgia Tech put email address screening in their second user survey and found that 4% of submissions were multiple submissions from the same address [8]. Problems endemic to traditional survey methods also exist online. Individuals may not be truthful about who they are. Those solely interested in obtaining an incentive may not complete a survey carefully. Research suggests respondents asked to identify themselves on a mailed print questionnaire may be more motivated to do a more accurate job of filling out the form, although not necessarily a more thorough job [6]. Offering an incentive requires requesting identifying information. This conflicts with the otherwise anonymous nature of the survey. Infomediaries [4]software agents for brokering personal information between online buyers and sellerscould remove this concern. However, when questions asked for free text response, no difference was found between electronic and paper groups [5].
Researchers must keep practical and ethical issues in mind when searching for survey respondents over the Internet. Respondents must be carefully, appropriately, and respectfully targeteda responsible approach that also happens to produce better results. Posting requests on newsgroups is less effective than a less intrusive approach using banner ads and search engines, where only those interested will notice the request. Such variations are consistent with traditional research methods: more appropriate audiences respond at a greater rate, as do those offered incentives. Contacting only the appropriate individuals and making it worthwhile to them will improve your chance of successfully communicating your message.
References
1. Comley, P. The use of the Internet as a data collection method. 1996, Paper presented at a joint EMAC/ESOMAR symposium on Technology for the New Marketing; www.virtualsurveys.com/papers/email.htm (visited Feb. 17, 2001).
2. Cranor, L.F. and LaMacchia, B.A. Spam! Commun. ACM 41, 8 (1998), 7483.
3. Haar, S.V. Click-through rates in a slide. Inter@ctive Week online. (Nov. 23, 1998); www.hdnet.com/intweek/stories/news/ 0,4164,2167752,00.html (visited Apr. 23 1999).
4. Hagel, J. Singer, M. Net worth. Harvard Business School Press, Boston, 1999.
5. Keisler, S., Sproull, L. Response effects in the electronic survey. Public Opinion Quarterly 50 (1996), 402413.
6. McDaniel, S.W. and Rao, C.P. An investigation on respondent anonymity's effect on mailed questionnaire response rate and quality. Journal of the Market Research Society 23, 3 (1981), 150160.
7. Michalak, E.E. The use of the Internet as a research tool: The nature and characteristics of seasonal affective disorder (SAD) amongst a population of users. Interacting with Computers 9, 4 (1998), 349365.
8. Pitkow, J. Recker, M. Using the Web as a survey tool: Results from the Second WWW User Survey 1995; www.igd.fhg.de/www/www95/ papers/79/survey/survey_2_paper.html (visited Feb. 17, 2001).
9. Pitkow, J.E. and Kehoe, C.M. Emerging trends in the WWW user population. Commun. ACM 39, 6 (1996), 106107.
10. Press, L. Resources for networks in less industrialized nations. IEEE Computer 28,6 (1995), 6671.
11. Reid, B. USENET Readership summary for Jul 1995, Posted to news.groups on Aug. 6, 1995, archived at www.tlsoft.com/arbitron/ jul95/arbitron.summary.txt (visited Feb. 17, 2001).
12. Strauss, J. Early survey research on the Internet: Review, illustration and evaluation 1996, Presented at the American Marketing Association 1996 Winter Educators' Conference: Marketing Theory and Applications; unr.edu/homepage/jstrauss/surveyresearch.html (visited Feb. 17, 2001).
Footnotes
This work was part of Tanya Cheyne's Diploma in Applied Psychology while studying at the University of Nottingham. This work was funded and supported by Design Interpretive at Nortel Networks.
Tables
Table 1. Responses to newsgroup postings by target group. "Hits" signifies the percentage of responses that fell into the target catagory.
Table 2. The response rate by target groups indicating the possible effect of incentives. Estimates are in brackets.
Table 3. Responses to banner advertisements by target group.
Table 4. Responses from search engines by target group. (Respondents could belong to more than one group.)
©2001 ACM 0002-0782/01/0400 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2001 ACM, Inc.
No entries found