The award-winning research team developed innovative algorithms for a mathematical approach called an "implicit solver" to realistically simulate Earth features at unprecedented resolution and accuracy. The team was able to predict the motions of the Earth's plates and the forces acting on them while also simulating the flow of mantle. Remarkably, the simulation involved more than 600 billion nonlinear equations, a major milestone in computational science and engineering
The simulations were performed on Sequoia, which consists of 96 IBM BlueGene/Q racks, reaching a theoretical peak performance of 20.1 petaflops. Each rack consists of 1,024 computer nodes, hosting 16 core POWER processor chips designed for Big Data computations that are running at 1.6 GHz.
The team's code reached an unprecedented 97 percent parallel efficiency in scaling the solver to 1.6 million cores, a new world record. This milestone was achieved by rethinking the end-to-end computational framework, from the mathematical model to the numerical algorithms to the massively parallel implementation. The team devised a numerical algorithm that could tackle the vast range of scales present in Earth's mantle while also mapping efficiently to the massively parallel architecture of the BlueGene/Q supercomputer.
"These advances will open the door to addressing such fundamental questions as what are the main drivers of plate motion and what are the key processes governing the occurrence of great earthquakes", stated Prof. Michael Gurnis, Director of the Seismological Laboratory at the California Institute of Technology.
"While the conventional view was that efficiently solving highly nonlinear equations on millions of cores would be intractable, we demonstrated that with careful redesign of discretization, algorithms, solvers and implementation, it would be possible", stated Prof. Georg Stadler of New York University's Courant Institute of Mathematical Sciences.
"This work has application to a much broader class of models in science and engineering involving complex multiscale behaviour", stated Omar Ghattas, Director of the Center for Computational Geosciences at the Institute for Computational Engineering and Sciences and Professor of Geological Sciences and of Mechanical Engineering at the University of Texas at Austin.
Despite being an underlying cause for devastating earthquakes, volcanoes and tsunamis, many of the fundamental principles behind the flow of the mantle remain a mystery to scientists. In fact, understanding this "mantle convection" has been designated as one of the "10 Grand Research Questions in Earth Sciences" by the U.S. National Academies.
"We are only beginning to demonstrate how the combination of advanced algorithms, supercomputing and Big Data drawn from sensors and the Internet of Things can realistically simulate the most extreme nonlinear, heterogeneous forces of nature", stated Costas Bekas, manager of Foundations of Cognitive Solutions, IBM Research - Zurich. "We are exploring new ways to apply the availability of tremendous amounts of field sensor data and cognitive computing to a topic and then enable practitioners to reduce the time to solution from years to weeks and even days for everything from inventing a new material to discovering an untapped source of energy."
Authors of the paper detailing the work include:
IBM scientists have now been awarded the Gordon Bell Prize six times, most recently in 2013.
The team's research was reported in the SC15 paper, titled "An Extreme-Scale Implicit Solver for Complex PDEs: Highly Heterogeneous Flow in Earth's Mantle".