Communications of the ACM
Professor Jack Dongarra Announces New Supercomputer Benchmark
"The relevance of the Linpack as a proxy for real application performance has become very low, creating a need for an alternative," says Linpack creator Jack Dongarra of the University of Tennessee.
Credit: University of Tennessee
The way the power of supercomputers is measured is about to change.
Since 1993, Jack Dongarra, distinguished professor of computer science at the University of Tennessee has led efforts to rank the world's top 500 supercomputers. The much-celebrated bi-annual TOP500 list is compiled using Dongarra's benchmark system, called Linpack. It is the most widely recognized and discussed metric for ranking high-performance computing systems.
But Dongarra says Linpack hasn't kept pace with supercomputing needs and must be updated.
"Linpack rankings of computer systems are no longer so strongly correlated to real application performance," Dongarra says.
Dongarra and his colleague Michael Heroux, from Sandia National Laboratories in Albuquerque, New Mexico, are developing a new benchmark that is expected to be released in time for the next TOP500 list release in November.
Linpack measures the speed and efficiency of linear equation calculations. Over time, applications requiring more complex computations have become more common. These calculations require high bandwidth and low latency, and access data using irregular patterns. Linpack is unable to measure these more complex calculations.
"This is an important issue to address since we are seeing more applications being dominated by differential equations, and thus, each iteration of the TOP500 will show increasing gaps between real versus Linpack performance," Dongarra says.
The new benchmark, called the High Performance Conjugate Gradient (HPCG), should better correlate to computation and data access patterns found in many applications today. This could improve the high performance community, because some of today's supercomputers are being designed to gain a high score for the TOP500 list than to meet real-life demands.
"We have reached a point where designing a system for good Linpack performance can actually lead to design choices that are wrong for the real application mix, or add unnecessary components or complexity to the system," Dongarra says. "The hope is that this new rating system will drive computer system design and implementation in directions that will better impact performance improvement for real applications."
Dongarra also expects the new benchmark to adapt to emerging trends, unlike its predecessor.
"The Linpack benchmark is an incredibly successful metric for the high-performance computing community," Dongarra says. "The trends it exposes, the focused optimization efforts it inspires, and the publicity it brings to our community are very important. Yet the relevance of the Linpack as a proxy for real application performance has become very low, creating a need for an alternative."
Linpack will not be laid to rest, however. Instead, HPCG will serve as an alternative ranking of the TOP500 list, allowing a re-ranking of the systems on the list to "real" applications.
The next Top500 list will be announced at SC2013, the International Conference for High Performance Computing, Networking, Storage and Analysis, to be held in November in Denver, Colorado.
No entries found