Communications of the ACM
Home/Magazine Archive/April 2006 (Vol. 49, No. 4)/Find that Photo!: Interface Strategies to Annotate.../Full Text
Supporting exploratory search
Find that Photo!: Interface Strategies to Annotate, Browse, and Share
As digital photos become the standard media for personal photo taking, supporting users to explore those photos becomes a vital goal. Dominant strategies that have emerged involve innovative user interfaces that support annotation, browsing, and sharing that add up to rich support for exploratory search. Successful retrieval is based largely on attaching appropriate annotations to each image and collection since automated image content analysis is still limited. Therefore, innovative techniques, novel hardware, and social strategies have been proposed. Interactive visualization to select and view dozens or hundreds of photos extracted from tens of thousands has become a popular strategy. And since the goal of photo search is to support sharing, storytelling, and reminiscing, experiments with new collaborative strategies are being examined.
While digital photographic databases and retrieval systems have been in use for many years, these systems were typically designed for professionals in museums, libraries, advertising, and journalism, to name a few specíálitíes. Such systems employed a cadre of financially motivated individuals to hand-annotate the pictures with metadata such as keywords, dates, and locations, often using fixed vocabularies, to support traditional search techniques. By contrast, consumers typically put little effort into photo annotation; they are more focused on exploratory search and serendipitous discovery of photos with a stronger emphasis on entertainment. This leads to a very different set of requirements for personal photo use where ease of annotation, support for exploratory browsing, and convenient sharing is crucial.
Annotate. In textual exploratory search, users can enter key phrases from a document to retrieve similar content. But for images, retrieval based on content through automated analysis is often limited to some forms of shape analysis (such as finding the presence of faces in an image) and color matching to find sunrises or determine whether an image was taken inside or outside.
To support effective exploratory search on photos, appropriate annotations must be associated with the images either by the camera or by users of the images, such as the photographer or potentially a larger community of users. Cameras are increasingly recording information about the photograph including time and date stamps, tilt sensors for orientation, light levels, focal distances, and even global position. Barcodes, RFID tags, or other labeling methods could enable a higher percentage of photos to be annotated automatically.
Many interfaces enable manual annotation of photographs by "painting" keywords [3] or dragging and dropping names onto images. Commercial tools such as Adobe PhotoShop Album make tags drag-able onto photo borders. Other tools perform temporal clustering to create a more manageable set of photo groups [1]. As with many tasks, manual annotation can be improved by designing interfaces that support faster and easier annotation as well as making the future benefits more apparent. Automatic and manual annotations are valuable in supporting both searching and browsing.
Consumers typically put little effort into photo annotation; they are more focused on exploratory search and serendipitous discovery of photos with a stronger emphasis on entertainment.
Browse. Users browse for fun and to find a specific photograph. They may be looking for photos of their grandfather, their hike down the Grand Canyon, or a friend's wedding. They also may be looking for a great photo to accompany a story of a sunrise hike or memorable baseball game.
Clearly, if the photo collection has been extensively annotated, techniques such as faceted search (see Hearst's article in this section) can help users filter down a collection and show potential targets for browsing. User-controlled visualization of photos grouped by date, location, or annotation can greatly facilitate browsing and increase enjoyment [4]. Different layouts of photos can exploit this metadata to help people find desired photos and discover new ones. In particular, geo-tagging of photos and interfaces, like WWMX, allows people to find all those photographs associated with a particular area (see Figure 1).
Chronological displays work well for dates as well, but large numbers of photos can be overwhelming, so groups of photos can be clustered by date and representative photos can be manually or automatically chosen for each cluster [1, 2]. These representative photos again help to provide landmarks in order for users to locate photos from particular events. Interfaces such as PhotoMesa use powerful filtering tools, plus flexible grouping and rapid zooming, to enable users to explore thousands of photos fluidly (see Figure 2).
Share. Sharing photos by email, instant messaging, Web sites, and cell phones is a growing success story. When users select photos and make them available to others, they seem to be willing to invest more effort in annotation. Also by making them public, they invite others to comment and add annotations. More elaborate story-generating tools invite users to provide slideshow sequences with text captions and audio narration.
Recent innovations in social experiences on the Web have sought to encourage annotation by increasing satisfaction and making the benefits immediately apparent. A game-like approach to image annotation gets players to cooperate with anonymous, remotely located partners in assigning keywords for photographs [5]. This surprisingly addictive game has succeeded in labeling over 10 million images as of August 2005 (since its introduction in 2003). Other communities, such as Flickr, allow users to share and annotate images on a Web site using tags. These "folksonomies" have now gone past photos to Web pages and blogs as well (such as technorati and deli.cio.us).
The trend toward annotating, browsing, and sharing your photos via Web sites such as Flickr, Ofoto, and Shutterfly is perhaps one of the biggest changes enabled by the transformation from analog to digital photography. Photos no longer sit unattended in shoeboxes stored in attics, but are available for ready viewing by friends and family distributed around the world.
Summary
A combination of annotation, browsing, and sharing of photos can support the special exploratory search needs of personal digital photo users by getting around the fact that direct search of image content continues to be beyond the capabilities of current systems.
The special needs of amateur digital photographers are pushing the photo industry to support users with their desired activities. Social networking, in combination with innovative user interfaces and visualization, is just beginning to support everyday photographers. However, we see significant work remaining, especially in metadata standardization to help users cope with their rapidly growing and increasingly valued collections.
References
1. Graham, A., Garcia-Molina, H., Paepcke, A., and Winograd, T. Time as essence for photo browsing through personal digital libraries. In Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries (2002). ACM Press, NY, 326335.
2. Huynh, D., Drucker, S., Baudisch, P., and Wong, C. Time Quilt: Scaling up zoomable photo browsers for large, unstructured photo collections. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2005). ACM Press, NY, 19371940.
3. Kuchinsky, A., Pering, C., Creech, M., Freeze, D., Serra, B., and Gwizdka, J. FotoFile: A consumer multimedia organization and retrieval system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (1999). ACM Press, NY, 496503.
4. Kustanowitz, J. and Shneiderman, B. Meaningful presentations of photo libraries: Rationale and applications of bi-level radial quantum layouts. In Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (2005). ACM Press, NY, 188196.
5. van Ahn, L. and Dabbish, L. Labeling images with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2004). ACM Press, NY, 319326.
Figures
Figure 1. The WorldWide Media Exchange (WWMX) interface showing map and calendar views along with images as published in ACM Multimedia 2003; wwmx.org.
Figure 2. PhotoMesa showing 114 photos in six groups in a single view with integrated annotation and search tools as published in ACM UIST 2001; (courtesy of www.photomesa.com).
©2006 ACM 0001-0782/06/0400 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.
No entries found