Communications of the ACM

Technical opinion

Web Site Success Metrics: Addressing the Duality of Goals

By France Belanger, Weiguo Fan, L. Christian Schaupp, Anjala Krishen, Jeannine Everhart, David Poteet, Kent Nakamoto
Communications of the ACM, December 2006, Vol. 49 No. 12, Pages 114-116
10.1145/1183236.1183256
Comments

The Internet's ascension from an obscure U.S. Department of Defense experiment to a cultural icon has been remarkable. In less than a decade, it has extended into nearly every facet of society, from commerce to education to gaming. As of August 2006, there were approximately 92 million Web sites [3]. With the sheer quantity of sites, evaluating their success has added importance, but determining appropriate metrics to measure success isn't a trivial task.

Previous research on Web site success has often taken the perspective of the user, more specifically looking at consumer adoption of e-commerce Web sites. However, Web sites will have various goals, and success is clearly linked to achieving these goals. In addition, users and Web site owners may have different goals as well, and unless there is convergence of these goals, success will be evaluated differently by the two parties. Since users view the organization through its Web site and the organization represents itself through its Web site, we argue that successful Web sites should be designed to address the multiple goals of the owner while taking into account the multiple audiences of the site. The better a site can help the convergence of the goals of the users and the owners, the more successful the site will be. These perspectives are captured in the following Web site success considerations:

• Web site success is goal specific. Web site owners have many objectives for their sites, such as selling, informing, or advertising. These objectives have to match the site design. For example, for Amazon.com, the main goal may be to sell products and/or maximize profit. Selling products requires attractive design features and transactional capabilities (among others) while reducing costs to maximize profits may require self-service capabilities on the site. Conversely, Google.com's goal may be to enable end users to quickly identify relevant information to attract more users, thereby increasing its market share and improving its brand equity.
• Web site success is audience specific. Users have several motivations for going to Web sites, from browsing, entertaining, to acquiring goods and services. For a given site, a user may be positively influenced to return by the Web site's advanced customization capabilities, while another may be negatively affected by the lack of privacy that results from those same customization capabilities. The first user may wish to transact rapidly, not worrying about information given to the site, while the second might desire to browse the site more anonymously.
• Web site success takes into account diverse perspectives: users vs. owners. In this column, we argue there is a duality of goals based on perspectives of end users and Web site owners. The perspective then is linked to varying ideas of success. For the same site, these two perspectives may not match and measuring success from each perspective may mean different things. For example, for a user looking for product information, such as the content of a book from Amazon.com, success would be whether the information is found and the search experience was positive. However, from the firm perspective (Amazon.com), the goal is maximizing profits. Success is accordingly measured by net profits. Since users and organizations take different stands regarding success, metrics need to take both perspectives to gain a balanced view.

Back to Top

Web Site Goals Taxonomy

To begin an investigation of Web site success, the organization must first identify its goals, audiences, and the motivations for its audiences accessing the site. Researchers (for example, [2]), however, often classify sites based on functionality: online storefronts, Web presence sites, content sites, malls, incentive sites, and search agents. The table here presents a new way to view Web sites based on a set of goals.

In practice, two sites may share similar functions but have different goals, and therefore different definitions of success. Our taxonomy is a starting point for an investigation of success based on hypothesized goals. Each of our categories should therefore be sufficiently distinct that they'll have some goals that are unique to that classification while not being specific to one particular Web site type. Each Web site can combine multiple classifications from our taxonomy for its many different audiences. For example, "informed decision-biased" Web sites can also be "e-commerce." Then, the goals and success measures from these classifications will be combined.

Back to Top

Measuring Web Site Success

Measuring success is difficult because of the various perspectives taken, as outlined previously. Taking the end user perspective, it is essential that prior expectations are met and the user leaves the site satisfied. In order to meet consumer expectations, a minimum level of criteria must be met. There are also factors whose presence will lead to satisfaction, but the absence of which won't lead to dissatisfaction. These are known as "enhancing factors" [6], which provide "extra" satisfaction beyond what is expected. Finally, there are success factors which, if not delivered will cause dissatisfaction, but if delivered above a certain level can enhance satisfaction [6]. The predominant way of determining success from the user's perspective is through surveys, where users' satisfaction and likelihood of return are of interest. Taking the organization's perspective, success would be the site's ability to create an ongoing relationship with users, which will either immediately or eventually lead to additional visits by the user or transactions to be conducted. Gathering clickstream data and making inferences from site traffic is the predominant way of determining success from organizations' perspective. However, organizational metrics to measure site effectiveness need to be tied to specific goals.

Factors identified in previous research as determining success include: site quality, information quality, and net benefits [1, 4, 5]. We suggest two additional factors of importance for Web site success: system quality and image. System quality is concerned with whether there are "bugs" in the Web site's underlying information system, in addition to security, responsiveness, and overall reliability. Site quality is concerned with the user interface, overall ease of use, navigability, searching capabilities, and customization features. Information quality is concerned with issues such as accuracy, understandability, informativeness, and relevance of information generated by the system via the Web site. Image is concerned with the Web site and organization's overall reputation. Net benefits, adapted from Seddon's definition of net benefits [5], represents in our context the sum of all benefits (past and expected future) minus all costs (past and expected future) that can be attributed to visiting the Web site. In order to determine the net benefits, one must adopt some stakeholder's point of view (user vs. organization) about what is valuable and what isn't [5].

The opposing success metrics that can result for the different perspectives for two types of Web site goals are illustrated in the figure here. As can be seen, some metrics hold true across different Web sites goals as well as across perspectives. However, some success factors differ depending upon the Web site goal and perspective taken. In order for a Web site to be deemed successful there must be a match between the firm's Web site objectives, the user's goals when using the Web site, and the Web site's design. Therefore, one way to identify design features is to map Web site goals to consumer goals. Then, one should critically evaluate whether the back-end metrics, or the measures of the system itself, are in line with the front-end metrics, or the visible measures of success. For example, is the way the system is designed consistent with traditional measurement methods for success? If a consumer's interaction goal is to rapidly gain relevant information, a site with many pictures but no appropriate search tool will result in a mismatch, and therefore an unsatisfied user who will never return.

Back to Top

Conclusion

To design success metrics, we must first understand the idiosyncrasies of the various goals and audiences of Web sites. Without a clear understanding of these goals, we cannot generalize findings on success from a study of one Web site to another. Therefore, we lay the foundation toward the objective of measuring success with a taxonomy of Web site goals and their corresponding success metrics. Only a few specific metrics were listed, and now a systematic empirical study of metrics for each Web site goal that takes into account perspectives of both end users and organizations is needed. Once completed, the detailed Web site goal taxonomy and success metrics should make the measurement of Web site success more successful.

Back to Top

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.

---

No entries found