Communications of the ACM
Wiring Reconfiguration Saves Millions For Trinity Supercomputer
Cabinets for the Trinity supercomputer at Los Alamos National Laboratory.
Credit: Los Alamos National Laboratory
A moment of inspiration during a wiring diagram review has saved more than $2 million in material and labor costs for the Trinity supercomputer at U.S. Department of Energy's Los Alamos National Laboratory.
The Laboratory's High Performance Computing (HPC) facilities team, led by Data Center Manager Ron Velarde of HPC-Design (HPC-DES), discovered the potential to re-engineer Cray's initial wiring diagram for the power feed to Trinity's computing racks. As a result, the facilities team was able to perform a redesign, approved by Cray and the Laboratory, that ultimately saved the Lab an estimated $2.6 million in material and labor costs. Sandia National Laboratories adopted this redesign for its Trinity-like systems and Lawrence Berkeley National Laboratory is considering it as well.
"My team and I are always looking to improve engineering means and methods for supercomputer installations in our data centers," Velarde says.
Trinity, Los Alamos's latest major high-performance computer, is designed to provide increased computational capability for the NNSA Nuclear Security Enterprise in order to improve geometric and physics accuracy in calculations that can be completed in weeks — not years.
The facilities team also used prefabricated copper tray cable rather than fabricating the 22,000 feet of cables on site, which was the plan originally proposed for the project. For high-performance computing systems in the Laboratory Data and Communications Center, where power cables are being routed over the computers for the first time, the facilities team worked with the Lab's electrical standards personnel to approve new aluminum power cables, saving 20 percent in materials cost and a factor of three in weight. This pioneering use of aluminum is also expected to generate cost and weight savings in other areas of the Laboratory in addition to the high-performance computing data centers.
The Trinity Phase 1 system, based on Intel Corp.'s Haswell processor, is currently performing simulations. The Trinity Phase 2 system, based on Intel's Knight's Landing processor, is being delivered this summer. As an example of the scales involved, Phase 2 of the Trinity system, which began to arrive in June, weighs just under 100 tons. Trinity in its completed form will weigh about 175 tons and contain twelve miles of copper cable and 44 miles of optical cable. More than 16,000 gallons of water will circulate in the inner process loop to cool Trinity at a flow rate of around 10,000 gallons per minute.
HPC Team
The team that performed the work includes Velarde, Andres Borrego, Michael Ferguson, Alynna Montoya-Wiuff, Eloy Romero, and Loren Serna, all of the Los Alamos HPC-DES group. The NNSA and Advanced Simulation and Computing Program funded the work, which supports the Lab's Nuclear Deterrence mission area and the Integrating Information, Science and Technology for Prediction science pillar.
No entries found