Shaheen recently completed a set of ultra-resolution subsurface mapping and characterization simulations for the Geophysics Technology Division at the Saudi Aramco EXPEC Advanced Research Center. For the first time, scientists produced a 3D image of the subsurface geologic layers to a 7.5 meter resolution.
"Unlocking ultra-resolution subsurface mapping and characterization at production scale is a major milestone for exploration", stated Ali A. Al-Meshari, manager of EXPEC ARC. "This is a major step toward seamless integration of geophysical, geological and engineering models."
To put the team's accomplishment in perspective, it allows scientists to see underground features half the length of a semi-trailer or lorry; one-third the length of a bowling alley; or about three times longer than a sedan car.
Aramco scientists used integrated GeoDRIVE - Geophysical solution for Dynamic Real-time Image and Velocity Estimation - software to successfully produce a 3D image of subsurface geologic layers at a record resolution of 7.5 meters at production scale. Previous simulations had a resolution of tens of meters.
"The vast computational cost of accurate seismic simulations created a gap between theoretical advancements in the academic world and practical industrial applications", stated Ali Momin, GeoDRIVE project lead. "This provided us with an unique opportunity to build an innovative solution that can satisfy our current needs as well as the future needs of researchers and professionals."
Supercomputers consist of tens of thousands of CPUs, memory chips, kilometers of networking cables, thousands of disks and hundreds of applications and libraries. Shaheen, which is located in the KAUST Supercomputing Core Laboratory (KSL), is one of the largest pure CPU systems in the world with a total of 12,348 Intel Haswell CPUs, 790 TB of memory and 17.6 PB of storage. It is currently the largest supercomputer in the Middle East.
Campaigns like the one conducted by the Aramco team require upward of 54,000 simulations using more than 100,000 cores, or around half of the total capacity of Shaheen. Large scale campaigns - referred to as hero runs - require careful optimization of the system.
One key element of a successful hero run is the readiness of the system. A single underperforming component can jeopardize the performance of the entire run. Maciej Olchowik, the systems administrator team lead, and Bilel Hadri, a computational scientist in the KSL, checked the health of all Shaheen components using a special test suite they developed before handing over the system to Aramco scientists.
KSL scientists also helped to optimize the simulations so that Aramco scientists can explore high-fidelity resolutions and more diverse scenarios.
"My mission is to help scientists use our resource in the most efficient way", stated KSL computational scientist Samuel Kortas. "That is why I developed the software tool Decimate - it makes complex things as simple as possible for end users."
With the Decimate tool, Aramco scientists were able to test all the possible combinations of cache usage patterns by exploring large numbers of parameter values. The optimum pattern found with the tool was used for the final production run.
This is the third world record resulting from the productive collaboration between Saudi Aramco and the KAUST Supercomputing Core Lab. The teams previously collaborated to conduct the largest-ever scale reservoir simulation and the largest-ever scale engineering simulation.
"The KAUST Core Lab maintains world-class state-of-the-art facilities, and I feel that Shaheen, as a national asset, should be further utilized to meet the demands of organisations around the Kingdom", stated Justin Mynar, KAUST associate vice president for research and executive director of the KAUST Core Labs.