Communications of the ACM

Communications of the ACM

Populating the Hermitage Museum's New Web Site

By Fred Mintzer, Gordon W. Braudaway, Francis P. Giordano, Jack C. Lee, Karen A. Magerlein, Silvana D'Auria, Amnon Ribak, Gil Shapir, Fabio Schiattarella, John Tolva, Andrey Zelenkov
Communications of the ACM, August 2001, Vol. 44 No. 8, Pages 52-60
10.1145/381641.381657
Comments

Russia's palatial State Hermitage Museum in St. Petersburg possesses many great collections [10] including prehistoric art, antiquities, Western European art, arms and armor, oriental art, and numismatics. Indeed, the Hermitage's collections are vast, numbering over three million holdings. Arguably, it is the largest art gallery in the world. Although more than 500 rooms are open to the public, they display less than 10% if its holdings.

In 1997, IBM and the Hermitage began exploring the nature of museum information services that would best serve the museum. Underlying these discussions was the vision of Mikhail Piotrovski, director of the Hermitage, that information services could help fulfill the museum's mission in the 21st century. Three end-user applications were identified for development: multimedia-based art education housed in an education and technology center, visitor information kiosks, and a new Web site that would permit the Hermitage's collections to be searched and better experienced from afar.

In addition, an image creation studio would be developed to produce high-quality digital images for the three applications. The project was briefly described in [7] and www.hermitagemuseum.org/html_En/00/hm0_8.html. Its initial deployment was unveiled on June 15, 1999 (see www.ibm.com/news/1999/06/15.phtml)

The multimedia-based education programs were created to serve Hermitage visitors—especially children—by providing interactive art education. They run on seven workstations in an Education and Technology Center (ETC) within the museum. Initially, each workstation offered two courses developed cooperatively by Hermitage and IBM educational personnel; one featuring art on the life of Jesus, the other featuring antiquities. Much of the material in each course is presented by displaying high-resolution images of art with a narrative describing the artist, the art's theme, and other interesting aspects. A variety of quiz-type games are included.

The visitor information kiosks were created to help Hermitage visitors choose a set of holdings to visit and then navigate the museum in order to see them. Visitors may select a general interest suggested tour of specified duration or a museum collection, or prepare a personalized tour; six suggested tours and 31 tours of museum collections are offered. Alternatively, the visitor may compose a personalized tour by leafing through a set of Hermitage highlights and selecting up to five. To facilitate navigation, each kiosk plots the tour route on maps of the Hermitage's interior. Thumbnail images of prominent Hermitage features, serving as navigational milestones, are superimposed on the maps.

Figure 1. Exterior view of the Hermitage. Used by permission of the State Hermitage Museum, St. Petersburg, Russia.

Here, we use the term "museum collections" to indicate an organization of holdings by region and era of origin, as is done in the museum community. We use the term "tour" to describe the route to the holdings of interest, and not a person-hosted tour.

Figure 2. The Small Throne Room, with painting Peter I with Minerva by Jacopo Amigoni. Used by permission of the State Hermitage Museum, St. Petersburg, Russia.

The Web site was created to serve remote patrons by familiarizing them with the Hermitage's holdings, its history, news, temporary exhibitions, and the visitor services it provides. Remote patrons are served in two languages: English and Russian. The quality of its images was intended to permit visitors to experience the museum's art. A digital library of images and descriptive text adds the ability to search the Hermitage's holdings.

Back to Top

The Web Site

The Web site (www.hermitagemuseum.org) represents the collective expertise of a large technical team that developed the system, a large team of museum professionals that developed procedures for data development and validation, and a large design team responsible for the aesthetics and usability of the site. It also represents the dedicated labor of many professionals who populated the site with text and images. More than 100 technical and Hermitage museum professionals contributed to the site.

The Web site consists of two nearly independent parts that serve different purposes. The static part consists of over 1,500 pages of detailed descriptions of museum collections ("Collection Highlights"). Like an art catalog, it reflects museum collection structure and is illustrated by well-known highlights. It rarely changes. The dynamic part consists of a digital library of holdings ("Digital Collection") accessible in many ways. It is updated often.

Prior to the Hermitage/IBM project, the museum provided an attractive Web site presenting many of the museum's collections. The static part of the current Web site was based on the design and content of this prior site. An international team led by IBM e-business services in Atlanta structured, designed, and implemented the static part. Its creation began with development of a comprehensive set of standards for the graphical design of the pages and for the ways information could be presented. Next, the graphical elements were completed. Images and descriptive text were created. Pages were then assembled and tested in English. Finally, companion Russian-language pages were produced.

Many Hermitage specialists contributed to the data development process for the dynamic element of the site. This begins with selection of holdings, followed by their classification by a team of database and information administrators. Then, with a curator, the cataloging attributes (for example, technique, genre, epoch) are added. Concurrently, Hermitage writers develop annotations validated by a curator. When the holding's information is completed, it is translated into English. Photography of the holdings, scanning, and image preparation takes place at the same time. Lastly, a complete set of information is entered—in English and Russian—into the digital library.

It was anticipated that many site visitors might not be familiar with the organization of museum collections. To help them better browse the Hermitage's holdings, the dynamic part of the site organized the holdings in a different manner—as 12 digital collections whose contents depend on the form of the art's representation. The digital collections are:

• Paintings, prints, and drawings;
• Sculpture;
• Machinery and mechanisms;
• Arms and armor;
• Furniture and carriages;
• Ceramics and porcelain;
• Applied arts;
• Jewelry;
• Textiles;
• Numismatics and glyphs;
• Costumes; and
• Archaeological artifacts.

Figure 4. Portrait of Lady-in-Waiting to the Infanta Isabella by Peter Paul Rubens. Used by permission of the State Hermitage Museum, St. Petersburg, Russia.

To browse, one first selects the desired digital collection; and collection-specific information is presented to help refine the search. Each collection has its own categories of cataloging information. "Artist's name" and "painting's title," for example, facilitate browsing through paintings, but would be of little use in browsing through archaeological artifacts.

Parametric searches on database fields within the digital library that describe the digital collections are also provided. Here, the visitor first selects a digital collection, then enters a description of the desired holdings. Although perhaps cumbersome for the casual visitor, these searches better support the Hermitage staff and the expert visitor.

The Web site also offers a free text-based quick search across digital collections. One might use this function, for example, to locate all works of Michelangelo (Buonarroti), including both sculpture and painting. We note that searching on "Michelangelo" locates the statue Crouching Boy by Michelangelo Buonarroti, a virtual tour of the Michelangelo gallery, and a painting by Michelangelo Merisi (commonly known as Caravaggio).

The Web site also supports searches on the paintings, prints, and drawings digital collection based on the visual layout (form and color) or color palette of holdings using IBM's Query By Image Content (QBIC) technology [3].

Having located a holding, the visitor can launch a similarity search for other holdings that share some characteristic. The supported similarity searches depend on the digital collection to which the located holding belongs. For example, the similarity search for paintings enables searching for other paintings by the same artist, of the same style, from the same country/region, made using the same technique, of the same genre, or with similar visual layout.

Creating images that permit the Web site visitor to experience the Hermitage's holdings is very challenging. The Web site provides visitors with high-resolution images carefully retouched by Hermitage staff. For each holding, two full-screen images are offered to the visitor. One is intended for Web site visitors using a display with 800 X 600 pixel resolution, the other for visitors using a display with 1,024 X 768 pixel resolution. They provide the largest images that will fit on the respective displays, given the normal amount of screen space occupied by the Web browser.

In addition, an interactive zoom function allows the visitor to select a region of an image and view that region with an increased level of detail. A coarse enlargement is first provided, then refined as additional information is received from the site.

Still images provide excellent representations of many of the Hermitage's holdings. However, the Hermitage's galleries challenge the limitations of still images. To enable Web site visitors to better experience Hermitage galleries, the site contains 14 panoramic tours of Hermitage rooms, galleries, and exterior views, presented with IBM's HotMedia technology [6]. To better represent Hermitage statues, multiple views are sometimes provided.

Back to Top

The Digital Library

The digital library enables the browsing and searching provided by the site. It generates dynamic pages for specifying browsing and searches, executes the searches, and prepares dynamic pages that present the search results.

Earlier we noted the Hermitage's holdings on the dynamic side were organized as 12 digital collections, each of which could be searched (or browsed) on descriptive information specific to that digital collection. The digital library stores each holding with cataloging information specific to its digital collection, a text object that combines the structured information, and visual features that enable the QBIC searches.

In addition, seven derivative images are stored for each holding, pregenerated to improve response time (versus generation on the fly). Created from the holding's scan, the derivative images satisfy different application needs. They are:

• A small thumbnail used to present search results;
• A larger thumbnail used in pages that provide "more information about an artwork";
• A quarter-screen image describes the holding and permit zoom regions to be specified;
• A full-screen image for display at 1024 X 768 pixel resolution;
• A full-screen image for display at 800 X 600 pixel resolution;
• A high-resolution image from which the zoom's detail is composed; and
• A scan-resolution image not used by the Web site that may be manually extracted for use by kiosks, ETC programs, or other applications.

IBM's Digital Library product (since renamed Content Manager) provides the digital library's core information management functions. For this project, it was enabled to support the Russian character set. Application code was developed to generate the dynamic pages that permit queries to be specified and results presented. This was a significant task because of the variety of queries offered and the need to support them in both Russian and English.

Application code was also written to enter Hermitage content into the digital library. Each digital collection has its own data model that specifies how that collection's information is structured in the digital library (structured fields, objects and their relationships). Each digital collection requires a loader to collect the information associated with the holding, perform consistency checking on it, and enter it into the digital library (in batch mode). The development of a composite loader that supports the 12 data models in English and Russian was a significant project task.

Several data-entry applications were written to facilitate secured data entry. Dictionaries were used by the data-entry applications to better manage entries for some information categories. The use of dictionaries eased the translation burden, since a repeated entry would not require a new translation when encountered again. It also increased cataloging consistency. For example, as holdings painted by Rembrandt (Harmensz Van Rijn) were loaded into the digital library, his name would be extracted from the same dictionary entry, ensuring all instances would use the same representation.

Back to Top

The Image Creation Studio

The Image Creation Studio (ICS) permits Hermitage staff to scan its holdings, retouch the images, and prepare derivative images for the Web site. Furthermore, the ICS integrates these functions into a unified workflow.

Image capture is provided by an IBM Pro/3000 scanner [4]. Other Pro/3000 scanners have been used in earlier cultural heritage projects, often to capture images of homogeneous collections of materials. In the IBM/Vatican Library project [8], for example, they captured predominately images of manuscript pages. The Hermitage's holdings, by contrast, are very diverse. Some are photographic, some are not; some are transmissive, some reflective; some are 2D, some 3D; and they are of varying size. Developing illumination, fixtures, and workflow needed to create images from the diversity of Hermitage holdings was another significant task.

Figure 5. Ptolemy II and Arsinoe II (The Gonzaga Cameo), Ancient Egypt, 3d century BC. Used by permission of the State Hermitage Museum, St. Petersburg, Russia.

Significantly less retouching is required when a holding can be moved to the ICS and captured directly (versus scanned from a photograph of the holding). Several lighting and mechanical fixtures were developed to enable direct capture, including reflective halogen lighting for photographic prints and drawings, reflective fluorescent lighting for coins and medallions, a lighting booth for small 3D holdings, and a book easel for holding bound books.

However, it is often not practical to move holdings to the studio because of their size, fragility, or sheer value; they must first be photographed, then scanned from the photographic materials. This procedure was the dominant form of image capture despite the advantages of direct capture.

To achieve the highest quality, the scanned images must be retouched. Many holdings, such as oil paintings, have shiny surfaces that produce reflections that should be removed. Photography distorts the color of the holdings. Scanned holdings often exhibit dust. Both Adobe PhotoShop and Micrographx Picture Publisher were used to perform the retouching.

The ICS prepares the derivative images for the Web site from the retouched images using a batch process. For each source image, derivatives are created by: reducing the source image; sharpening the reduced-size image; color transforming the reduced image to the color space of a standard monitor [8]; embedding an invisible watermark (optional); and compressing the resulting image.

Larger derivatives are generally created first and intermediate reductions are retained to serve as source images for the creation of smaller derivatives (to reduce computations). A color management system helps retain the color content throughout the processing.

To better protect the Hermitage's intellectual property, invisible watermarking was applied to larger derivative images made available through the Web site. The watermark cannot be seen by a human observer, but can be extracted by a computer program. If an image, perhaps used commercially, is suspected of being Hermitage property, a watermark extraction is attempted; an extracted watermark provides evidence of Hermitage ownership.

The watermarking technique [1] used embeds watermarks that are resistant to many types of image processing [2]. We note the watermark may be extracted from printed material by first scanning the material and then running the extraction. Furthermore, each watermark is embedded with a companion watermark key that functions much like a password; the watermark cannot be extracted without it. Hence, a malicious party who tried to remove the mark without the watermark key would not know if the removal attempt was successful. Detection of the mark, following a removal attempt, would always be a possibility.

The watermarking process inserts a small texture into the source image. Extraction is performed by computing a number of statistics that compare the texture of a candidate image with the inserted texture. A visualization of the watermark extraction is composed by exclusive oring of the statistics with an institutional logo. An example source image, watermarked image, and visualization of an extracted watermark are shown in Figure 3.

The diversity of the visual content of Hermitage's holdings proved challenging to the watermarking. Some images, especially large bright images with little texture, were successfully marked at low watermarking strength. Other smaller, darker, or more textured images required higher watermarking strength. To accommodate this diversity, the strength was determined adaptively. For each holding, one of the derivative images was marked and extraction attempted. If unsuccessful, the strength was increased and the process repeated; when successful, the other derivative images were marked at this strength.

The three ICS functions—scanning, retouching, and derivative-image creation—are often performed by different personnel; the ICS's workflow provides that the output of each ICS function may be stored on CD-ROM to decouple the functions.

The ICS workflow also ensures that vital information about the images entered during scanning is retained through later stages. During the scanning process, the operator enters an annotation that uniquely identifies the holding; this is preserved in the image's header. This unique identifier is also entered in the cataloging information describing the holding. The loader later uses this information to match the holding's images with its cataloging information. The scan application also enters color (calibration) information into each scanned image's header.

The unique identifier and color information would normally be removed by the retouching software. Prior to retouching, the ICS workflow reads the image headers and saves this information; it also rotates the images to the proper orientation and corrects their appearance for the monitor on which they are viewed. After retouching, annotations and color calibration information are replaced in the image headers. The derivative image creation retains the identifiers and color information throughout its processing so they will be available to the loader.

Back to Top

Emerging Issues

Several of the authors have collectively participated in the development of other information systems for cultural institutions [5, 7, 8]. In this project, some new issues were addressed; some old issues were addressed in new ways.

Protecting online intellectual property from misappropriation is a continuing issue for online digital libraries. In this project, it was a matter of greater concern because the Web site made images available at high resolution (and consequently high value). However, the visible watermarking technology used in earlier IBM projects [5, 8] was judged inappropriate for art. When darkly applied, it interfered with the visitor's experience of the art; when lightly applied, it left the visitor wondering what was painting and what was watermark. This project was the first in which we relied on invisible watermarking to protect images made available online.

One of the project's great challenges was diversity—of applications, of holdings, and of user communities. The diversity of applications required a significant software development effort. The diversity of holdings inspired the development of 12 data models and a composite loader to support them. It inspired the development of scanning fixtures and a workflow to support capture of many types of objects. It inspired the kiosks and Web site to provide panoramas and multiple views of statues. It inspired a watermarking application process that adapted watermark strength to image content.

The diversity of user communities motivated the project to develop multiple applications in multiple languages. It motivated the Web site to support two sizes of full-screen image for user communities with different display resolutions. It motivated the project to develop multiple search techniques—browsing on collection-specific parameters and free text-based quick searches for the general user, parametric searches for the expert user, and visual searches for those most interested in visual appearance. Often, the result of dealing with the diversity was to permit the user communities to view the same source content in different ways through different applications, translations, search interfaces, and image resolutions. Providing systems that more easily create multiple views of the same content is a worthy goal for future systems.

Dealing with diversity can have its benefits. Preparing content for one application, for example an ETC course, entails performing much of the work (research, image creation, data entry) needed to prepare it for others. This speeds both the creation of content and gives it greater value.

Another of the project's great challenges was size—the size of the museum's collections and the size of the user communities. Although images of fewer than 3,000 holdings were provided when the Web site was unveiled, this is a relatively large number of high-quality images for a museum site.

It is difficult to create large volumes of high-quality images. Scanning is one of the less laborious operations. Retouching, inspection, and researching the holding's cataloging are more time-consuming chores. There is a great need for technology that streamlines digital content creation through faster capture, inspection, and retouching; much progress is needed to make truly large collections available online.

Back to Top

Retrospective

In the days following the unveiling of the new Web site, it received nearly a million hits a day. Subsequently, it's averaged over 200,000 hits a day. In the year following the unveiling, it received Advanced Imaging magazine's "Solution of the Year" award for digital photography and still-image databasing, was voted "Best Site Overall," by the Museums and the Web 2000 conference, and received the Russian Internet Academy's Grand Prix award for best overall Web site in Russia. It has been covered in the New York Times, the Financial Times, and National Geographic Traveler.

However, the e-commerce community has shown little interest to date in replicating its function. In the museum community, there is a steady desire to provide a better experience of their collections; a more immersive experience is desired. Some of the authors are now exploring more realistic representations of 3D objects.

In the commercial realm, creating a fuller experience of merchandise has often been a lesser concern than image data size and scanning cost. This site may just be ahead of its time.

Interest in invisible watermarking continues to grow. However, we have come to appreciate that watermarking algorithms alone do not provide a good watermarking system. Proper support of watermarking also requires secure key generation and management; such a system has been a focus of some recent work.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2001 ACM, Inc.

---

No entries found