Communications of the ACM

Designing for the mobile device: experiences, challenges, and methods

Contextual and Cultural Challenges For User Mobility Research

By Jan Blom, Jan Chipchase, Jaakko Lehikoinen
Communications of the ACM, July 2005, Vol. 48 No. 7, Pages 37-41
10.1145/1070838.1070863
Comments

Personal, mobile synchronous and asynchronous communication has proven to be very desirable for all types of users, with estimates of more than a half-billion mobile phones sold each year [2]. Nokia Research Center's User Experience Group, working with other user practitioners in Nokia, seeks to understand why people do the things they do with their mobile communication devices and proposes solutions that best address their wants and needs. In accordance with the principles of user-centric product concept design [4], the solutions are designed to inform and inspire the product creation process within Nokia. Given that the group works three to eight years ahead of what appears on the market, confidentiality concerns restrict disclosure of many of these concepts and the findings on which these concepts are based. Instead, in this article we share some of the essential lessons learned from our projects centering on the early stages of product concept development. Two areas have proved to be particularly challenging in this respect: coping with multiple contexts and multiple cultures in the study of mobility.

Back to Top

Mobility Across Contexts and Cultures

The purpose of discovery research is to explore, document, and understand the scope of a particular theme and to use the findings of this research to guide and support the development of new products, applications, and services in the mobile realm. A product that benefited from applying such an approach is presence-enhanced contacts, a service incorporated in a number of popular Nokia models, including the 6170 handset sold in the U.S. Our recent discovery research themes include what people carry and why, time and contact management issues, communication needs of mature users, device competence of non-literate users, device sharing, and personal identity concerns. The topics we research are often quite broad, leading to a number of challenges in the methodologies we use. Here, we highlight these issues and how we have addressed them.

Contextual approach to mobility research. Mobile phones are carried and used in a wide range of environments, including at home, at work, on the street, while driving a car or using public transportation, and in restaurants or other public settings; literally from the moment people wake up to the moment they go to sleep. Tamminen et al. [8] state that mobile urban environments differ from indoor contexts, such as offices or lecture rooms in many important ways. Internal factors, such as tasks and goals are different due to external factors such as social resources that are dynamic and unpredictable. We fully agree with the authors and believe that studying the user in an everyday, mobile environment is a challenge.

Moving between environments, especially the transition from public to private spaces and vice versa, involves significant changes in context. The mobile phone is often used during these changes in environment and this too needs to be studied. For example, a phone call can be initiated on public transportation but completed after arriving home. To understand the user's context we need to gather data wherever the users are, and adapt to their changes in context. The methods we commonly use to this end include Contextual Inquiry, shadowing (essentially following people with their permission), short photo diaries, and observations. Changes in environment present a number of practical hurdles.

At night a shadowed participant can move from a well-lit home to a dark street to a nightclub. Flash photography draws unwanted attention that affects behavior and disturbs others in proximity, which is why it is banned from our research. Fortunately, night-vision-equipped cameras largely solve this problem. In close proximity public environments, such as a subway or bus, obvious use of the recording equipment again draws attention to the participant so the researcher is required to be adept at nonchalantly recording data of the user. The equipment can facilitate this—either by being hidden, disguised, or more often—by appearing not to be in use. Recording sufficient photo and video data in very tight environments requires the attachment of a wide-angle lens and is still impractical at short notice or for short periods of time. The length of time spent in proximity of our participants makes shooting video the whole time impractical: we work to the rule of thumb that an hour of video will take at least three hours to analyze. In many discovery projects capturing high-quality stills supported by short clips of video has proved sufficient. Research covering key-press input with the mobile phone definitely requires video and our colleagues have developed a video-camera kit that can be mounted on most mobile phones and accurately records both screen and input data [6, 7]. Lastly, there is no substitute for the effectiveness of a notepad and pencil.

Being prepared. Exploratory fieldwork requires researchers to plan for a range of outcomes. Unpredictable weather can make operating equipment difficult and create discomfort for the researchers. A rained-on participant can change into fresh clothes at home but the researchers are largely limited to what they are able to carry. Some items of note in the researchers' carried "emergency kits" include: coins and notes in the local currency; energy bars and drinks; pens; power plug adapter; spare batteries (for phone, laptop, camera); extra memory cards; DV cassettes; mini-tripod; clean clothing; rainproof outer shell; credit card (accepted in that culture), pre-paid travel card(s), and a printed copy of pertinent contact information.

Length of the session. The mobile phone allows participants to make ad-hoc changes to their plans. Given this variability, we reserve at least one full day to study any given individual. During screening we aim to understand the kinds of things the participant typically does during the week, to prepare for the shadowing session. Since interesting behavior can occur during movement between environments, participants are asked in advance about their travel plans, and the research team ascertains whether it is feasible to travel with the participant.

Gathering data on participants' communication activities during the shadowing session is challenging. The participant often receives phone calls or is intensively involved in text messaging dialogue. Such situations enable observation of the behavior at the one end of the conversation only. In order to get more fully into the communication activity, the researcher needs information regarding the context of the party not present in the situation: where is this person located and in what type of a situation? How are the two persons linked to each other? What motivates the communication?

The least intrusive way to elaborate on the communication activity is to retrospectively ask the participants to describe it in their own words. They do not necessarily have information about the context of the other party, but will be able to expand on the content and intertextuality of the communication incident. In the case of text-based communication, such as text messaging or mobile instant messaging, participants may be willing to show the dialogue to the researchers in addition to verbal description. Recording and logging communication activities during the shadowing would of course provide an effective means to shed light on what can normally only be observed. In addition to being associated with privacy implications, legal and ethical considerations make this data collection method a less viable choice.

Long durations of time spent in close proximity to the participant can create tensions related to privacy and personal space that are less apparent during shorter sessions. Although never completely removed, these tensions have been alleviated by recruiting through our personal network of contacts; arranging informal meetings prior to the data collection time; advocating a proactive privacy policy and updating this policy in the field to cover additional parameters as circumstances allow; data consent forms signed after the participants have reviewed data about themselves; and shadowing time-outs. Over the course of the day all participants need space for personal affairs and at such points participants may want to be as far away from researchers and their recording equipment as possible. Although these sensitive moments never constitute the focus of our research, they may overlap with issues that are relevant to us. For example, some people send text messages, talk, or read while using the bathroom. The solution in this example is to provide participants with tools for self-documentation where they can decide what level of intimacy is revealed. Choosing self-documentation over direct observation has been successfully used, for example, in research concerning text messaging behavior of teenagers [3].

Shadowing in moving vehicles. Where participants go we go, so understanding how they plan to travel is of importance. All modes of transportation present problems, although travel on foot is normally the easiest to deal with. In Tokyo during rush hour, for example, we managed to lose a subway-bound male participant during shadowing [1], because we were unable to follow him through a ticket barrier reserved for a particular combination of tickets. Longer train journeys require the researchers to stay awake in spite of the motion and warmth of the train, sleeping fellow passengers, and other factors. During another study of commuter use of WLAN services on trains in Japan, participants and researchers reported being affected by motion sickness.

When studying participants using bicycles, a researcher is required to have at least the same level of fitness as the participant, must be confident cycling around a (foreign) city, and must acknowledge the fact that capturing accurate data will be difficult. Cyclists are treated differently in different cultures and the research team has had to cope with cycling on pavements in Japan, loose interpretation of road rules by car drivers in Shanghai, and the relative serenity of dedicated cycle lanes in Berlin [1].

In a car, will there be enough room, and if so where will the researcher sit? In a taxi, the participant may have to pay extra for an additional passenger, so it is important to let them know in advance that additional costs incurred due to your presence will be settled at the end of the study. The safety of our participants and research team is paramount but each environment comes with its own set of risks. Before the study we clearly state the parameters of our research, and that our insurance does not cover the participant. The safety of the team is of particular concern during car travel—for example, if a participant decides to send a text message while driving.

Security. In response to commercial espionage, security and privacy concerns shops have their own set of strict rules about recording data and have sales staff and security guards to enforce these rules [1]. During a comparison of train station infrastructure in Mumbai, Hangzhou, Milan, and New York City, our researchers were challenged and asked to stop recording data both in Milan and New York City. Currently, the incidental video recording inside public buildings in the U.S. can be interpreted as possible terrorist-related surveillance activity: during a recent study in New York City a researcher was questioned by authorities and the photos were deleted.

Exploratory user research in the mobile realm presents many challenges including: the need to carry equipment that is suitable to collect quality data across multiple contexts; the ad-hoc nature of this kind of study; the long duration spent in the participant's proximity; and participant and research team security. The rewards for achieving the correct balance of these issues include rich insights into understanding user motivations and suitable material to inspire the concept design process.

Cross-cultural approach to mobility research. The half-billion phones sold in 2004 were purchased by consumers representing almost every country in the world. Our research covers a wide range of regions, from relatively mature mobile phone markets such as Sweden and the U.K., to emerging markets such as China, India, and Brazil (see Figure 1). The desire to localize products to these markets, the global nature of communication and vast cultural differences are acknowledged. However, there is also the need to adopt a design perspective that effectively balances the differences and similarities. The focus of our cross-cultural research is often in understanding the intersection of culture and technology adoption. We recently completed a large comparison study on mobile media use among European and Far East users [9]. Mobile phone use among non-literate users illustrates research into cultures situated at an earlier stage in the technological evolution.

Technological climate. One fundamental problem is in anticipating the technological climate in the regions of the forthcoming research. This becomes an issue with cross-cultural studies comparing use of particular applications and devices. Popular media tends to report on how state-of-the-art technologies enter the market. Sales figures on well-established technologies are more difficult to obtain, however, to some extent because large corporations often regard this kind of information as commercially sensitive. Consequently, making informed decisions about which countries to study is not a straightforward matter, even working with the resources of a global organization. The risk here is that insufficient market insight can lead to the adoption of inappropriate methodological tools, which, in turn, may decrease the sensitivity of the research.

Social acceptance. Being an obvious outsider can be a hindrance or a benefit to the data being collected depending on the context. Blending in is easier when a researcher is considered a "local," but outsiders tend to have more leeway in what is socially acceptable. Approaching people on the street to conduct an interview is easier when the person approaching appears not to be a local: it is an established norm for strangers to be asking directions from locals. We often emphasize the international nature of the research, so for example street interviews might start with "Hi, I'm running an international study in country x and y to study...".

Indirect research. Issues such as arriving at the participant's home on time, interacting with participants, or operating and understanding the device or user interface being studied can be difficult enough in one's own culture let alone in a different language and using novel devices. It is of course tempting to expand the network of researchers to outsource this kind of work to subcontractors. The crux of the problem is how to maximize the validity and reliability of the research when a local researcher is hired to moderate the interviews and sometimes even to analyze the data and generate reports. The ability to be in direct control of the data collection and analysis is important. If parts of this process are outsourced, the quality of the findings is at stake. This issue is prominent in the case of cross-cultural research, which requires high levels of investment.

In the mobile media study, we used focus groups as one of our data collection methods to understand how mobile media was used in various countries. Members from our own team were present in all of the sessions, but we hired local practitioners in each country to moderate the sessions and to produce a report on their respective regions. This process led to a number of insights:

• Cross-cultural research generates vast quantities of data. The importance of certain elements in the data may only be known months or years after collection. Should a need to do further analysis on the data arise at some later point, specifying the format(s), directory structure and file naming conventions for all raw data helps subsequent retrieval processes.
• Validity of cross-cultural research can be improved by being explicit and clear about the aims of the project and by making sure the questions asked are comparable across cultures. For instance, clearly formulated research questions will be of formative help to the data analysis process.
• Studying different cultures can lead to the identification of universal behavioral patterns. Researchers should also allow for country-specific methods and findings. Each culture will have its nuances and peculiarities when it comes to mobile device use, leading to windows of opportunity for design (see Figure 2).

The examples listed here show that all parts of a cross-cultural research project are associated with challenges. During the planning stage, making informed decisions about the target cultures is not always possible. Consequently, being able to select the appropriate set of data collection methods becomes difficult. When performing the actual studies, the use of local experts is often needed, potentially leading to validity and reliability problems. Conducting cross-cultural research is an expensive endeavor, raising the importance of fine-tuning the approaches used in accord with the lessons learned from the latest projects. Thus, methodological considerations and activities ought to take place after and between projects, as well. For instance, we actively develop, maintain, and integrate our network of local experts attuned to our research needs with each completed project.

Back to Top

Conclusion

To conduct corporate mobile phone user research means understanding our customer's context, which requires conducting cross-cultural studies in unique and challenging locations. The flip side to this is that the pace is fast, design solutions need to be proposed swiftly, and it is often difficult to adopt the kind of academic rigor these investigations might deserve. The challenge areas discussed in this article are not an exhaustive list and are not fully representative of the kinds of data collection methods we use. Overall, the issues probably raise more questions than they provide answers. What can be reliably concluded, however, is that rich contextual user research provides a meaningful framework on which to build the concepts for future applications.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.

---

No entries found