Communications of the ACM
Researchers Model Birth of niverse in Large Cosmological Simulation
This image shows a substructure within a galaxy halo in the Q Continuum simulation, with "subhaloes" marked in different colors.
Credit: Katrin Heitmann, et al.
Researchers are sifting through an avalanche of data produced by one of the largest cosmological simulations ever performed, led by scientists at the U.S. Department of Energy's Argonne National Laboratory.
The simulation, run on the Titan supercomputer at DOE's Oak Ridge National Laboratory, modeled the evolution of the universe from just 50 million years after the Big Bang to the present day — from its earliest infancy to its current adulthood. Over the course of 13.8 billion years, the matter in the universe clumped together to form galaxies, stars, and planets; but it's not clear precisely how.
These kinds of simulations help scientists understand dark energy, a form of energy that affects the expansion rate of the universe, including the distribution of galaxies, composed of ordinary matter, as well as dark matter, a mysterious kind of matter that no instrument has directly measured so far.
Intensive sky surveys with powerful telescopes, like the Sloan Digital Sky Survey and the new, more detailed Dark Energy Survey, show scientists where galaxies and stars were when their light was first emitted. And surveys of the Cosmic Microwave Background, light remaining from when the universe was only 300,000 years old, show how the universe began: "Very uniform, with matter clumping together over time," says Katrin Heitmann, an Argonne physicist who led the simulation.
The simulation fills in the temporal gap to show how the universe might have evolved in between: "Gravity acts on the dark matter, which begins to clump more and more, and in the clumps, galaxies form," Heitmann says.
Called the Q Continuum, the simulation involved half a trillion particles — dividing the universe up into cubes with sides 100,000 kilometers long. This makes it one of the largest cosmology simulations at such high resolution. It ran using more than 90 percent of the Titan supercomputer. For perspective, typically less than one percent of jobs use 90 percent of the Mira supercomputer at Argonne, according to officials at the Argonne Leadership Computing Facility, a DOE Office of Science User Facility. Staff at both the Argonne and Oak Ridge computing facilities helped adapt the code for its run on Titan.
"This is a very rich simulation," Heitmann says. "We can use this data to look at why galaxies clump this way, as well as the fundamental physics of structure formation itself."
Analysis has already begun on the two and a half petabytes of data that were generated, and will continue for several years, she says. Scientists can pull information on such astrophysical phenomena as strong lensing, weak lensing shear, cluster lensing, and galaxy-galaxy lensing.
The code to run the simulation is called Hardware/Hybrid Accelerated Cosmology Code (HACC), which was first written in 2008, around the time scientific supercomputers broke the petaflop barrier (a quadrillion operations per second). HACC is designed with an inherent flexibility that enables it to run on supercomputers with different architectures.
Details of the work are described in "The Q Continuum Simulation: Harnessing the Power of GPU Accelerated Supercomputers," published in the Astrophysical Journal Supplement Series by the American Astronomical Society. Other Argonne scientists on the study included Nicholas Frontiere, Salman Habib, Adrian Pope, Hal Finkel, Silvio Rizzi, Joe Insley, and Suman Bhattacharya, as well as Chris Sewell at DOE's Los Alamos National Laboratory.
This work was supported by the DOE Office of Science (Scientific Discovery through Advanced Computing jointly by High Energy Physics and Advanced Scientific Computing Research) and used resources of the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory (OLCF), a DOE Office of Science User Facility. The work presented here results from an award of computer time provided by the Innovative and Novel Computational Impact on Theory and Experiment program at the OLCF.
No entries found