Communications of the ACM
Communications of the ACM
Digital Experience
We experience our physical environment through our natural senses of sight, sound, touch, taste, and smell. Combined with the models of the world each of us develops through learning, they allow us to experience and function in the physical and social worlds. The history of civilization follows the development of our understanding of "experience" and how to share it with our fellow humans immediately, as well as with those who will follow in future generations (see the Symbolic Timeline). Experience is fundamental to human existence. The desire to share it will continue to be the motivating factor in the development of exciting multimedia technology in the foreseeable future.
Data is observed facts or measurements; information is derived from data in a specific context. Experience is the direct observation or participation in an event. A look at history reveals how human society has evolved into an information society and is on its way to being an experience society.
The Internet represents a major revolution in the way people interact with one another and store, access, share, and manipulate data. People read news in newspapers, not on TV, and see news on TV, not in newspapers. We want to read some things and experience others. In 1967, Marshal McLuhan, the media scholar and professor at the University of Toronto, declared, "The medium is the message." Yet the medium was the message when only one medium could be used to communicate the message. Now, the Internet allows the synthesis and rendering of information and experiences using whatever is the most appropriate media to convey the message. The message is just the message, and the medium is just the medium.
Toward omnipresence. Virtual reality is commonly used to provide the experience of "being there." Virtual reality systems require a detailed model of the environment in order to render that environment in response to user actions. In video games, the system models the environment and lets the user interact with it as if it were really there.
"Real reality" is the next step toward allowing users to experience and interact with real environments through all our human senses. For example, you might experience your friend's wedding in India, seeing what is happening, feeling the warm, humid air of the wedding hall, listening to conversations and the wedding music, and enjoying the taste and aroma of the food being served. You might experience all that and more while sitting at home in Montana on a frigid January morning.
Table 1. Major communication inventions and their effects on human society.
The Web today is a web of documents, though it's on its way to being a web of events I call EventWeb. You read documents and experience events. Soon, you will experience events posted on the Web, including past ones. You'll experience them from your own personal perspective, rather than from that of the author or producer. EventWeb will grant you omnipresence.
Compelling experiences. Compelling and engaging experiences require immersion in a rich set of data and information in a way that allows you to observe a subset of the data and information directly. If the data changes in response to your action, you're a participant. You'll even manipulate actions in the event-environment and change the nature of the event.
To create immersive and interactive environments, data has to be collected through multiple perspectives and multiple sensors. The complete environment is represented at the level of granularity required while capturing all the attributes relevant to human sensibilities. The attributes related to the primary senses—touch, taste, smell, sight, and sound—are acquired through a variety of sensors and stored in some kind of database. Sensors for sight and sound are fairly well developed and may soon surpass human capabilities. Sensors for touch, taste, and smell will soon receive attention from engineering researchers and sensor manufacturers and will be available commercially within 10 years. Thus, it is likely that the environment model in the next few decades may contain the kind of information that engages the basic human senses, as well as other information not directly sensable by humans but that may be used to enhance the experience.
Presentation of experience. Compelling experiences rely on carefully staged presentation. Immersed in an environment, you might want to focus only on a subset of sensed data. A good movie director knows in which sequence to present the elements of a story, as well as the most effective perspective, camera angles, and locations. A strong presentation system would allow casual users to determine the sequence in which they might want to experience an event and present the experience through appropriate devices.
Experience systems will create a network of real-life events experienced by users, much like Web users today immerse themselves in the network of documents. A user would be able to access any event captured on the Web anywhere and might want to experience it while monitoring others and even reviewing similar older events.
In the coming years, we'll see tremendous progress in presentation techniques related to all our senses. In fact, we may also start thinking about simulating other senses, mapping them to our existing ones to enrich our experience. The idea of mapping one sense to others may be a commonplace opportunity, opening our minds to many interesting experiences. Imagine smelling pictures and tasting videos.
Many scholars and engineers view Johannes Gutenberg's moveable type and printing press as the most important invention of the last millennium. Its influence has certainly been profound enough that each aspect of our society has been altered by it.
Digital experience is a major but natural next step in the evolution of technology affecting every aspect of human society, from education to sexual behavior, to business organization and operations, to health care. Today's digital experience will ultimately yield an experience society.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2001 ACM, Inc.
No entries found