The latest simulations, which mimic possible large-scale seismic activity in the southern California region, were done using a new software system called EDGE, for Extreme-Scale Discontinuous Galerkin Environment. The largest simulation used 612,000 Intel Xeon Phi processor cores of the new Cori Phase II supercomputer at the National Energy Research Scientific Computing Center (NERSC), the primary scientific computing facility for the Office of Science in the U.S. Department of Energy (DOE).
SDSC's ground-breaking performance of 10.4 PFLOPS (Peta FLoating-point Operations Per Second, or one quadrillion calculations per second) surpassed the previous seismic record of 8.6 PFLOPS conducted on China's Tianhe-2 supercomputer. Through efficient utilization of the latest and largest supercomputers, seismologists are now able to increase the frequency content of the simulated seismic wave field.
Obtaining higher frequencies is a key to predict ground motions relevant for common dwellings in conducting earthquake research. SDSC and Intel researchers also used the DOE's Theta supercomputer at the Argonne National Laboratory as part of the year-long project.
"In addition to using the entire Cori Phase II supercomputer, our research also showed a substantial gain in efficiency in using the new software", stated Alex Breuer, a postdoctoral researcher from SDSC's High Performance Geocomputing Laboratory (HPGeoC) and lead author of the paper, to be presented in June at the International Super Computing (ISC) High Performance conference in Frankfurt, Germany. "Researchers will be able to run about two to almost five times the number of simulations using EDGE, saving time and reducing cost."
A second HPGeoC paper submitted and accepted for the ISC High Performance conference covers a new study of the AWP-ODC software that has been used by the Southern California Earthquake Center (SCEC) for years. The software was optimized to run in large-scale for the first time on the latest generation of Intel data centre processors, called Intel® Xeon Phi x200.
These simulations, also using NERSC's Cori Phase II supercomputer, attained competitive performance to an equivalent simulation on the entire GPU-accelerated Titan supercomputer. Titan is located at the DOE's Oak Ridge National Laboratory and has been the resource used for the largest AWP-ODC simulations in recent years. Additionally, the software obtained high performance on Stampede-KNL at the Texas Advanced Computing Center at the University of Texas at Austin.
Both research projects are part of a collaboration announced in early 2016 under which Intel opened a computing centre at SDSC to focus on seismic research, including the ongoing development of computer-based simulations that can be used to better inform and assist disaster recovery and relief efforts.
The Intel Parallel Computing Center (Intel PCC) continues an interdisciplinary collaboration between Intel, SDSC, and SCEC, one of the largest open research collaborations in geoscience. In addition to UC San Diego, the Intel PCC at SDSC includes researchers from the University of Southern California (USC), San Diego State University (SDSU), and the University of California Riverside (UCR).
The Intel PCC programme provides funding to universities, institutions, and research labs to modernize key community codes used across a wide range of disciplines to run on current state-of-the-art parallel architectures. The primary focus is to modernize applications to increase parallelism and scalability through optimizations that leverage cores, caches, threads, and vector capabilities of microprocessors and coprocessors.
"Research and results such as the massive seismic simulation demonstrated by the SDSC/Intel team are tremendous for their contributions to science and society", stated Joe Curley, senior director of Code Modernization Organization at Intel Corporation. "Equally, this work also demonstrates the benefit to society of developing modern applications to exploit power-efficient and highly parallel CPU technology."
Such detailed computer simulations allow researchers to study earthquake mechanisms in a virtual laboratory. "These two studies open the door for the next-generation of seismic simulations using the latest and most sophisticated software", stated Yifeng Cui, founder of the HPGeoC at SDSC and director of the Intel PCC at SDSC. "Going forward, we will use the new codes widely for some of the most challenging tasks at SCEC."
The multi-institution study which led to the record results includes Alex Breuer and Yifeng Cui; as well as Josh Tobin, a Ph.D. student in UC San Diegos Department of Mathematics; Alexander Heinecke, a research scientist at Intel Labs; and Charles Yount, a principal engineer at Intel Corporation.
The titles of the respective presentations and publications are "EDGE: Extreme Scale Fused Seismic Simulations with the Discontinuous Galerkin Method" and "Accelerating Seismic Simulations using the Intel Xeon Phi Knights Landing Processor". The work was supported by the National Science Foundation (NSF), SCEC, and the Intel PCC initiative.