Communications of the ACM
Virtual Detectives se Social Media to Study Global Fashion Trends
Researchers used Convolutional Neural Networks to map input from a 15 million person photo corpus.
Credit: Cornell University
Each day billions of photographs are uploaded to photo-sharing services and social media platforms, and Cornell University computer science researchers are figuring out ways to analyze this visual treasure trove through deep-learning methods.
Kavita Bala, professor of computer science; Noah Snavely, associate professor computer science at Cornell Tech; and Kevin Matzen have released their results in a new paper, "StreetStyle: Exploring World-Wide Clothing Styles from Millions of Photos."
"We present a framework for visual discovery at scale, analyzing clothing and fashion across millions of images of people around the world and spanning several years," Snavely says.
Bala says the group used deep learning to detect various attributes — the color or sleeve length of shirts, whether a person is wearing glasses or a hat, and so on — in millions of images.
"Using these detected attributes, we can then derive visual insight," Bala says. "For example, where in the world is wearing hats more common? At what time of the year? Which colors are more popular in summer versus winter? Our approach produces a first-of-its-kind analysis of global and per-city fashion choices and spatio-temporal trends."
The researchers looked specifically for fashion trends based on time and location. To eliminate all unrelated photos, the group first used facial-recognition technology to exclude photos that did not include people in them. Then, the group filtered the results to include photos that included the upper half of the body.
That left about 15 million photos. With the narrower selection of photos, the team developed an object-recognition program that recognized items of clothing. The program also learned a number of descriptors like sleeve length, color, and pattern.
Once the images were tagged, the group put the data through another program, this one to recognize patterns not in the clothes but in the data. That churned out information on what clothing items were being paired with what, which trends were popular in which areas, and how the trends changed over the three-year time period.
This research provides a look into cultural, social, and economic factors that shape societies and provides insights into civilization.
"The combination of big data, machine learning, computer vision, and automated analysis algorithms makes for a very powerful analysis tool in visual discovery of fashion and other areas," Matzen says.
No entries found