Communications of the ACM
High-Speed Transatlantic Network to Benefit U.S.-based Collaborators
Credit: iStockPhoto.com
Scientists across the United States will soon have access to new, ultra high-speed network links spanning the Atlantic Ocean, thanks to a project currently underway to extend ESnet (the U.S. Department of Energy's Energy Sciences Network) to London, Amsterdam, and Geneva. Although the project is designed to benefit data-intensive science throughout the U.S. national laboratory complex, heaviest users of the new links will be particle physicists conducting research at the Large Hadron Collider (LHC), the world's largest and most powerful particle collider. The high capacity of this new connection will provide U.S. scientists with enhanced access to data at the LHC and other European-based experiments by accelerating the exchange of data sets between institutions in the United States and computing facilities in Europe.
DOE's Brookhaven National Laboratory and Fermi National Accelerator Laboratory — the primary computing centers for U.S. collaborators on the LHC's ATLAS and CMS experiments, respectively — will make immediate use of the new network infrastructure, once it is rigorously tested and commissioned. Because ESnet, based at DOE's Lawrence Berkeley National Laboratory, interconnects all national laboratories and a number of university-based projects in the United States, tens of thousands of researchers from all disciplines will benefit as well.
The ESnet extension will be in place before the LHC at CERN in Switzerland — currently shut down for maintenance and upgrades — is up and running again in the spring of 2015. Because the accelerator will be colliding protons at much higher energy, the data output from the detectors will expand considerably — to approximately 40 petabytes of raw data per year compared with 20 petabytes for all of the previous lower-energy collisions produced over the three years of the LHC first run between 2010 and 2012.
The cross-Atlantic connectivity during the first successful run for the LHC experiments that culminated in the discovery of the Higgs boson was provided by the US LHCNet network, managed by the California Institute of Technology. In recent years, major research and education networks around the world — including ESnet, Internet2, California's CENIC, and European networks such as DANTE, SURFnet, and NORDUnet — have increased their backbone capacity by a factor of ten, using sophisticated new optical networking and digital signal processing technologies. Until recently, however, higher-speed links were not deployed for production purposes across the Atlantic Ocean — creating a network 'impedance mismatch' that can harm large, inter-continental data flows.
Evolving Model
This upgrade coincides with a shift in the data model for LHC science. Previously, data moved in a more predictable and hierarchical pattern strongly influenced by geographical proximity, but network upgrades around the world have now made it possible for data to be fetched and exchanged more flexibly and dynamically. This change enables faster science outcomes and more efficient use of storage and computational power, but it requires networks around the world to perform flawlessly together.
"Having the new infrastructure in place will meet the increased need for dealing with LHC data and provide more agile access to that data in a much more dynamic fashion than LHC collaborators have had in the past," says physicist Michael Ernst of DOE's Brookhaven National Laboratory, a key member of the team laying out the new and more flexible framework for exchanging data between the Worldwide LHC Computing Grid centers.
Ernst directs a computing facility at Brookhaven Lab that was originally set up as a central hub for U.S. collaborators on the LHC's ATLAS experiment. A similar facility at Fermi National Accelerator Laboratory has played this role for the LHC's U.S. collaborators on the CMS experiment. These computing resources, dubbed "Tier 1" centers, have direct links to the LHC at Europe's CERN laboratory (Tier 0). The experts who run them will continue to serve scientists under the new structure. But instead of serving only as hubs for data storage and distribution among U.S.-based collaborators at Tier 2 and 3 research centers, the dedicated facilities at Brookhaven and Fermilab will also be able to serve data needs of the entire ATLAS and CMS collaborations throughout the world. And likewise, U.S. Tier 2 and Tier 3 research centers will have higher-speed access to Tier 1 and Tier 2 centers in Europe.
"This new infrastructure will offer LHC researchers at laboratories and universities around the world faster access to important data," says Fermilab's Lothar Bauerdick, head of software and computing for the U.S. CMS group. "As the LHC experiments continue to produce exciting results, this important upgrade will let collaborators see and analyze those results better than ever before."
Ernst adds, "As centralized hubs for handling LHC data, our reliability, performance, and expertise have been in demand by the whole collaboration and now we will be better able to serve the scientists' needs."
Investment in Science
ESnet is funded by DOE's Office of Science to meet networking needs of DOE labs and science projects. The transatlantic extension represents a financial collaboration, with partial support coming from DOE's Office of High Energy Physics for the next three years. Though LHC scientists will get a dedicated portion of the new network once it is in place, all science programs that make use of ESnet will now have access to faster network links for their data transfers.
"We are eagerly awaiting the start of commissioning for the new infrastructure," says Oliver Gutsche, Fermilab scientist and member of the CMS Offline and Computing Management Board. "After the Higgs discovery, the next big LHC milestones will come in 2015, and this network will be indispensable for the success of the LHC Run 2 physics program."
This work was supported by the DOE Office of Science.
No entries found