Communications of the ACM
Ncar Announces Powerful New Supercomputer For Scientific Discovery
Scientists used a supercomputer to develop this 3-D rendering of a major thunderstorm that caused flooding west of Boulder, Colo. in July 2011.
Credit: David Gochis / NCAR
The National Center for Atmospheric Research (NCAR) announced the selection of its next supercomputer for advancing atmospheric and Earth science, following a competitive open procurement process. The new machine will help scientists lay the groundwork for improved predictions of a range of phenomena, from hour-by-hour risks associated with thunderstorm outbreaks to the timing of the 11-year solar cycle and its potential impacts on GPS and other sensitive technologies.
The new system, named Cheyenne, will be installed this year at the NCAR-Wyoming Supercomputing Center (NWSC) and become operational at the beginning of 2017.
Cheyenne will be built by Silicon Graphics International Corp. (SGI) in conjunction with centralized file system and data storage components provided by DataDirect Networks (DDN). The SGI high-performance computer will be a 5.34-petaflop system, meaning it can carry out 5.34 quadrillion calculations per second. It will be capable of more than 2.5 times the amount of scientific computing performed by Yellowstone, the current NCAR supercomputer.
Funded by the U.S. National Science Foundation and the state of Wyoming through an appropriation to the University of Wyoming, Cheyenne will be a critical tool for researchers across the United States studying climate change, severe weather, geomagnetic storms, seismic activity, air quality, wildfires, and other important geoscience topics. Since the supercomputing facility in Wyoming opened its doors in 2012, more than 2,200 scientists from more than 300 universities and federal labs have used its resources.
"We're excited to bring more supercomputing power to the scientific community," says Anke Kamrath, director of operations and services at NCAR's Computational and Information Systems Laboratory. "Whether it's the threat of solar storms or a heightened risk in certain severe weather events, this new system will help lead to improved predictions and strengthen society's resilience to potential disasters."
The NWSC is located in Cheyenne, and the name of the new system was chosen to honor the support that it has received from the people of that city. It also commemorates the upcoming 150th anniversary of the city, which was founded in 1867 and named for the American Indian Cheyenne nation.
Super Specs
The new data storage system for Cheyenne will be integrated with NCAR's existing GLADE file system. The DDN storage will provide an initial capacity of 20 petabytes, expandable to 40 petabytes with the addition of extra drives. This, combined with the current 16 petabytes of GLADE, will total 36 petabytes of high-speed storage. The new DDN system also will transfer data at the rate of 200 gigabytes per second, which is more than twice as fast as the current file system's rate of 90 gigabytes per second.
The system will include powerful Intel Xeon processors, whose performance will be augmented through optimization work that has been done by NCAR and the University of Colorado Boulder. NCAR and the university performed this work through their participation in the Intel Parallel Computing Centers program.
Even with its increased power, Cheyenne will be three times more energy efficient (in floating point operations per second, or flops, per watt) than Yellowstone, its predecessor, which is itself highly efficient.
"The new system will have a peak computation rate of over 3 billion calculations per second for every watt of power consumed," says NCAR's Irfan Elahi, project manager of Cheyenne and section manager for high-end supercomputing services.
Detailed Models
High-performance computers such as Cheyenne allow researchers to run increasingly detailed models that simulate complex processes and how they might unfold in the future. These predictions give resource managers and policy experts valuable information for planning ahead and mitigating risk.
Some of the areas in which Cheyenne is expected to accelerate research include the following:
Streamflow. Year-ahead predictions of streamflows and associated reservoir levels at a greater level of detail will provide water managers, farmers, and other decision makers with vital information about likely water availability and the potential for drought or flood impacts.
Severe weather. By conducting multiple simultaneous runs (or ensembles) of high-resolution forecast models, scientists will lay the groundwork for more specific predictions of severe weather events, such as the probability that a cluster of intense thunderstorms with the risk of hail or flooding will strike a county at a particular hour.
Solar energy. Specialized models of solar irradiance and cloud cover will be run more frequently and at higher resolution, producing research that will help utilities predict how much energy will be generated by major solar arrays hours to days in advance.
Regional climate change. Scientists will conduct multiple simulations with detailed climate models, predicting how particular regions around the world will experience changing patterns of precipitation and temperature, along with potential impacts from sea level rise, streamflow, and runoff.
Decadal prediction. Ensembles of detailed climate models will also help scientists predict the likelihood of certain climate patterns over a 10-year period, such as the risk of drought for a certain region or changes in Arctic sea ice extent.
Air quality. Scientists will be able to simulate the movement and evolution of air pollutants in far more detail, thereby better understanding the potential health effects of particular types of emissions and working toward improved forecasts of air quality.
Subsurface flows. More accurate and detailed models will enable researchers to better simulate the subsurface flows of water, oil, and gas, leading to a greater understanding of these resources.
Solar storms. Innovative, three-dimensional models of the Sun will lay the groundwork for predictions of the timing and strength of the Sun's 11-year cycle as well as for days-ahead forecasts of solar disturbances that can generate geomagnetic storms in Earth's upper atmosphere.
"Supercomputing is vital to NCAR's scientific research and applications, giving us a virtual laboratory in which we run experiments that would otherwise be impractical or impossible to do," says NCAR Director James Hurrell. "Cheyenne will be a key component of the research infrastructure of the United States through its provision of supercomputing specifically tailored for the atmospheric, geospace, and related sciences. The capabilities of this new system will be central to the continued improvement of our ability to understand and predict changes in weather, climate, air quality, and space weather, as well as their impacts on people, ecosystems, and society."
Key features of the new Cheyenne supercomputer system:
- 5.34-petaflop SGI ICE XA Cluster featuring a future Intel Xeon processor product family
- More than 4K compute nodes
- 20% of the compute nodes have 128-Gbyte memory and the remaining ~80% have 64-Gbyte memory
- 313 terabytes of total memory
- Mellanox EDR InfiniBand high-speed interconnect
- Partial 9D Enhanced Hypercube interconnect topology
- SUSE Linux Enterprise Server operating system
- Altair PBS Professional Workload Manager
- Intel Parallel Studio XE compiler suite
- SGI Management Center & SGI Development Suite
- Mellanox Unified Fabric Manager
The new Cheyenne supercomputer and the existing file system are complemented by a new centralized parallel file system and data storage components. Key features of the new data storage system:
- Four DDN SFA14KX systems
- 20 petabytes of usable file system space (can be expanded to 40 petabytes by adding drives)
- 200-Gbyte per second aggregate I/O bandwidth
- 3,360 × 8-Tbyte NL SAS drives
- 48 × 800-Gbyte mixed-use SSD drives for metadata
- 24 Network Shared Disk servers
- Red Hat Enterprise Linux operating system
- IBM General Parallel File System
No entries found