Communications of the ACM
Facial-Recognition Software Shows Promise Identifying Subjects of Portrait Art
Face-recognition technology under development at the University of California, Riverside recognized three-dimensional portraits of the same individual: a death mask (left) and bust (right) of 15th century Italian statesman Lorenzo de' Medici.
Credit: National Gallery of Art
The National Endowment for the Humanities has awarded researchers at the University of California, Riverside a $60,000 grant to continue their development of face-recognition software to help identify unknown subjects of portrait art.
A $25,000 grant in 2012 allowed the research team — Conrad Rudolph, professor of art history; Amit Roy-Chowdhury, professor of electrical engineering; and Jeanette Kohl, associate professor of art history — to begin establishing general parameters of the technology to recognize faces in portrait art. In one example of their success, the software was able to recognize two three-dimensional portraits of the same individual, a death mask and bust of 15th century Italian statesman Lorenzo de' Medici.
In the second phase of the project, "FACES: Faces, Art, and Computerized Evaluation Systems," the researchers will build on their initial successes to study the applicability of automated face-recognition technologies for analyzing portraits under different paradigms, including artist and period styles. They also will continue development of an algorithm robust enough to deal with the problems of angle views, aging, and personal artistic style that it can determine the likelihood of a probable match.
"Before the advent of photography, portraits were depictions of people who were important in their own worlds," says Rudolph, the project's principal investigator. "But, as a walk through almost any major museum will show, a large number of these unidentified portraits from before the 19th century — many of them great works of art — have lost the identities of their subjects through the fortunes of time."
Several dozen facial features are used in face recognition technology, Rudolph and Roy-Chowdhury say. Repeated testing and analysis in the first phase of the FACES project indicated that two key categories of information required for identification are local feature similarity, such as the corners of the eyes and the corners of the mouth, and anthropometric distance similarity, such as the width between the eyes and the width of the mouth. The team established 16 points of reference for the former, and 11 for the latter.
"While a lot of work has been done on face recognition in images, the application of this technology to the analysis of portraits provides a host of unique challenges and opportunities," Roy-Chowdhury says. "The methods developed need to be invariant to the style of specific artists, appropriate facial features have to be identified, and these need to be done with limited data that can be authenticated by experts like art historians. Once the methodologies have been developed, they can provide an independent and objective input to resolve some long-standing questions regarding the identities of subjects in ancient art works."
Technology that "reads" human faces already must contend with variations in facial expressions, age, facial hair, angle of pose, and lighting, Rudolph says. Refining that technology to recognize human faces in two- or three-dimensional art — as well as different artist and period styles — introduces further challenges, as does portrait art generally in that the image is not a photographic likeness, but rather one that is a visual interpretation on the part of the artist.
"It looks like you, but better," Rudolph says. "That makes it harder to use face-recognition technology, which doesn't deal with highly subjective images. This is a painstaking process."
During the second phase the researchers will analyze portraits and related material concerned with Dante, the famous Renaissance architect Brunelleschi, Lorenzo de' Medici, Henry VII, Anne Boleyn, Shakespeare, Isaac Newton, and many important papal portraits. They also will test a number of well-known but unidentified Renaissance portraits of women, crypto-portraits and crypto-self-portraits of Renaissance artists Mantegna and Michelangelo, self-portraits by Bernini, and a large body of portraits that seem to show the same Moche (Andes A.D. 100-800) noble throughout the course of his life.
"Many of these images are famous, and many of the identity questions are high-profile and so high-risk," Rudolph says. "Others are less well-known, like the Moche portraits, but are part of important issues in their own fields of study and also may receive a high degree of attention."
For example, the team hopes to examine several portraits that purport to be of Shakespeare and compare them with a portrait art historians know with certainty is the Bard of Avon.
If the refinement of facial recognition software to identify works of art is as successful as the researchers believe it will be, the technology could be used in museums and art conservation laboratories as a standard part of curatorial and preservation practice. It could also be used to recognize variations in architectural details that would reveal new information about building processes, building history, and architectural details, and might also have potential with paleography (ancient writing), possibly allowing the determination of the origin and date of thousands of ancient manuscripts for which identification remains a very subjective matter.
The researchers plan to develop a website and a museum exhibition to demonstrate the use of facial recognition technology to identify portrait subjects.
No entries found