Communications of the ACM
How to Fit a Planet Inside a Computer
Developing the Energy Exascale Earth System Model
The E3SM Earth system model can simulate storms with surface winds faster than 150 miles per hour.
Credit: LLNL
The Earth was apparently losing water.
Ruby Leung, a scientist from the U.S. Department of Energy's Pacific Northwest National Laboratory, and her team were baffled by their results.
"We were seeing the sea level decreasing at an alarming rate," she says.
Fortunately, they were only looking at a virtual Earth. They quickly realized there were errors in the Earth system computer model they were developing. Scientists use these computer programs to visualize the present and see into the future. They need to come as close as possible to modeling how Earth's systems function in real life. Because the real Earth cycles water but never loses it, neither should the model.
Identifying the problem was simple. Fixing it wasn't. There are hundreds of variables in the model that could affect its water cycle. Pinpointing the exact one and changing it without creating another inaccuracy can take hours or days of work. In this instance, the model wasn't sending all the water runoff fluxes from the land into the ocean. In addition, the atmospheric model was losing a very tiny amount of water at each time step, which added up to quite a lot. To fix the problem, Leung and colleagues changed the model to conserve water.
Solving this problem was just one of the many challenges the team faced as they developed DOE's new software: the Energy Exascale Earth System Model (E3SM). By representing many of Earth's systems and interactions in greater detail than ever before, they hope to help scientists better understand the planet today and in the future.
Accelerating Change
Earth always has been and always will be changing. But human activities are accelerating those changes and there is general agreement that global temperatures are rising. While scientists can broadly predict how rising temperatures will affect the world, policy-makers and planners need to understand what will happen to individual cities and states. That's where Earth system models come into play.
DOE is interested in understanding how energy production could affect Earth's systems and vice versa. While energy production can accelerate climate change, rising temperatures can also affect energy production and use. A lack of water can reduce a hydropower plant's output or limit water available for cooling nuclear plants.
"DOE and its predecessors have always had a mission to understand the environmental consequences of energy production and use," says David Bader, a scientist at DOE's Lawrence Livermore National Laboratory and E3SM team lead. "We know that the climate and Earth system is not stationary. It changes. We need a way to understand the interactions."
The scope of Earth system models makes them unique. Scientists construct a variety of software programs to simulate different natural systems, but Earth system models bring them all together. They demonstrate how these individual systems—the atmosphere, soil, ocean, ice sheets, and more—interact. They also incorporate many aspects of human activity, such as the products of energy production, water management and use, and agriculture management, as well as other land-use or land-cover changes.
These combinations help scientists understand the current and future interplay between the natural world and human activities. To ensure a model is correctly representing the present, they compare modeled results to real-life observations. The predictions looking decades into the future help planners of bridges or power plants understand how their long-term choices could affect or be affected by these changes.
Answering Questions
While there's only one Earth, there are many Earth system models. The E3SM team members designed their model to answer questions relevant to DOE's mission.
One of the holy grails when it comes to energy planning is being able to predict how and when rain, snow, and other precipitation fall over land. Climate and land use changes could increase the atmosphere's ability to hold moisture and cause storms to occur more frequently and intensely. In contrast, it could also lead to more evaporation, resulting in more drought. Or most likely, both could occur in different places around the world.
The new model focuses on three areas that have significant effects on surface precipitation, winds, and temperature as well as energy production: the water cycle, the way different Earth system components exchange biogeochemical fluxes, and the movement and melting of ice sheets.
"To predict precipitation, you need to really understand almost every other part of the Earth system," Leung says. "The atmosphere is playing the role of connecting everything. The atmosphere is not local."
Combining these systems into one model and getting the necessary level of detail isn't something that can be done on a desktop computer. Instead, it requires the big machines—the supercomputers at DOE Office of Science user facilities. E3SM programmers wrote the software to take full advantage of these computers' hardware and operating systems. The team is also designing it so they could upgrade the model to run on DOE's future exascale computers. These computers will be 50 times more powerful than today's fastest computers.
Using these computers, the model will be able to provide much more detail than previous Earth system models. The team expects E3SM will offer twice the level of overall detail using the Oak Ridge Leadership Computing Facility's new Summit supercomputer, compared to current models. Depending on the need, it can also offer the same amount of overall detail in most of the world, while providing extremely high resolution in specific areas. This regionally refined resolution will allow scientists to look at effects on a much more regional basis. That could help them better understand human activities, like farming, that vary from place to place.
This computing power will also allow them to run 10 times as many simulations as they can currently.
"We can now do more simulations using less resources," says Mark Taylor, a scientist at DOE's Sandia National Laboratories and E3SM Chief Computational Scientist.
Running the program on exascale computers will allow even bigger jumps in the model's complexity and detail.
Peering Into the Future
Building a model this complicated is like rebuilding a car. You might start with the same frame, but by the time you tune up some old parts, swap out others for new ones, and get the whole thing working together, it looks and acts completely differently.
To start off, the team worked with an existing Earth system model—the Community Earth System Model. The biggest challenge with themodel—and all existing Earth system models—is that there are areas where simulations are consistently inaccurate. While their modeling broadly matches up with the evidence, certain details don't quite correspond with observations.
Unfortunately, these problems don't have a simple fix.
"It's not like you can change one thing and it corrects. You change one thing and you usually make something else worse because the system is interconnected," says Bader. Just like the real Earth's system.
There are several pathways modelers are pursuing to reduce these errors.
Revising the base model is the most logical place to start. Scientists can often improve models by tweaking the physics equations and ecological theories that are based on research findings. Observational data, such as from the DOE Office of Science's ARM Climate Research Facility, regularly provide new insights.
Swapping out existing sub-models for new, more accurate ones is another approach. The E3SM team undertook the huge task of developing four separate, completely new sub-models representing rivers, the global ocean, sea ice, and land ice.
"There are processes that normally have not been represented in any Earth system model around the world," Leung says of the river model. "Without these parts, we are missing some of the important links in the global energy, water, and biogeochemical cycles."
But adding in these new models isn't a matter of cut-and-paste. Scientists need to connect them accurately so that changes in one correctly affect the others.
"You replace all of these components and you hope this model will be working better than before. But it's not pure luck. You really need to go back to basics," Leung says.
Similar to the "missing water" problem, the team faced a similar challenge with their ocean model. After they replaced it, they saw that the full Earth system wasn't simulating the El Nino-Southern Oscillation, a major influence on weather patterns. The team realized that the atmosphere and ocean models were representing the relationship between the wind and water movement in the ocean differently than observations did. To make both more accurate, they revised the processes to match up with each other.
Already Improving
All of this hard work paid off when the team released the first version of the model this past April. But they're far from done. In fact, they expect to produce at least three more versions in the future.
They're already improving the current model by modifying the sub-models within it. Researchers have published a paper, "Representing Floodplain Inundation in an Earth System Model," looking at how to improve the way the river model represents flooding in the Amazon. By using topographical data about river channels and information about how water flows between the land and river, they were able to make the sub-model more accurate. Another study, "Accounting for Groundwater Use and Return Flow Improves Modeling of Water Management," describes how the river model could better show how and when different sectors of society use surface water and groundwater.
At the same time, scientists are using the current model to run a series of experiments. As they finish, they'll release the model "data" on the DOE Earth System Grid Federation data archive of model results. The project has also made its code publically available through the popular programming website GitHub.
As the project continues, scientists are working to get a clearer glimpse into the future of the Earth.
"All of these models represent the cumulative knowledge that we've gained over the last 40 years," Bader says. "That's resulted in both a better understanding and therefore representation of both energy and Earth system processes. It's much more complete than previous models."
No entries found