Joined by representatives from the National Science Foundation (NSF) - which funded the system with a $60 million award - UT Austin, and technology partners Dell Technologies, Intel, Mellanox Technologies, DataDirect Networks, NVIDIA, IBM, CoolIT and Green Revolution Cooling, TACC inaugurated a new era of academic supercomputing with a resource that will help the U.S. top researchers explore science at the largest scale and make the next generation of discoveries.
"Scientific challenges demand computing and data at the largest and most complex scales possible. That's what Frontera is all about", stated Jim Kurose, assistant director for Computer and Information Science and Engineering at NSF. "Frontera's leadership-class computing capability will support the most computationally challenging science applications that U.S. scientists are working on today."
Frontera has been supporting science applications since June and has already enabled more than three dozen teams to conduct research on a range of topics from black hole physics to climate modelling to drug design, employing simulation, data analysis, and artificial intelligence at a scale not previously possible.
First announced in August 2018, Frontera was built in early 2019, and earned the no. 5 spot on the twice-annual TOP500 list in June, achieving 23,5 PetaFLOPS (23,5 thousand million million floating-point operations per second) on the high-performance LINPACK benchmark, a measure of the system's computing power.
Olexandr Isayev, a chemist from the University of North Carolina, used Frontera to run more than three million atomic force field calculations in less than 24 hours - a major achievement in high-speed quantum computation. The calculations are part of an effort to train an artificial intelligence system that can predict the likely characteristics of new drug compounds and identify compounds with the ability to target specific cells.
"It's a great machine, especially for quantum mechanics applications", Olexandr Isayev stated. "We're really looking forward to running large-scale calculations that were not possible before."
Ganesh Balasubramanian, an assistant professor of Mechanical Engineering and Mechanics at Lehigh University, has been using Frontera to study the dynamics of organic photovoltaic materials and model manufacturing conditions.
"The lightning speed at which Frontera performs computations is very beneficial", stated Ganesh Balasubramanian, who during the early user period experienced a five-time speed-up in his simulations of solar material manufacturing. "Overall, the entire pace of computational research will be increased by the arrival of Frontera."
Manuela Campanelli, an astrophysicist at the Rochester Institute of Technology, has been using Frontera to perform the longest simulations ever of the merger of neutron stars, including for the 2017 event detected by the Laser Interferometer Gravitational-Wave Observatory (LIGO), the Europe-based Virgo detector, and 70 ground- and space-based observatories.
"Frontera is an amazing system because it gives us a very large number of computer nodes that we can use to solve very complex problems", Manuela Campanelli stated. "These types of resources are unavailable on most university campuses, so we really need to have Frontera in order to be able to do the simulation we do."
Frontera combines Dell EMC PowerEdge servers with 8008 compute nodes, each of which contains two 2nd generation Intel Xeon scalable ("Cascade Lake") processors, totaling more than 16.000 processors and nearly half a million cores, connected by a 200 gigabit per second HDR Mellanox InfiniBand high-speed network.
The system incorporates innovative flash storage from DataDirect Networks and novel cooling systems from CoolIT, Cooltera, and Green Revolution Cooling (GRC), and employs several emerging technologies at unprecedented scale, including high-powered, high-clock rate versions of the latest Intel Xeon processors, Intel Deep Learning Boost, Intel Optane memory, and several kinds of liquid cooling.
In August, Frontera added two new subsystems to provide additional performance and to explore alternate computational architectures for the future. A 360 NVIDIA Quadro RTX 5000 GPU (graphics processing unit) system submerged in liquid coolant racks developed by GRC will explore more efficient ways to cool future systems, as well as explore single-precision optimized computing. An IBM POWER9-hosted system with 448 NVIDIA V100 GPUs will provide additional performance, and provide the top-performing GPU architecture with tight coupling to the processor and memory subsystems. These additional systems will accelerate artificial intelligence, machine learning, and molecular dynamics research for Frontera researchers in areas ranging from cancer treatment to biophysics.
In the coming months, Frontera will integrate with Cloud providers Microsoft, Google, and Amazon to provide researchers access to emerging computing technologies and long-term storage.
Frontera will operate for at least five years and will support hundreds of research projects and thousands of researchers in nearly every field of science over its lifetime. It is expected to have a major impact on:
Projects will be selected through a competitive application process and researchers will need to show that they require a computer at the scale of Frontera to solve their problems.
Faculty at the Oden Institute for Computational Engineering and Sciences at UT Austin are leading the world-class Frontera science applications and technology team, with partners from the California Institute of Technology, Cornell University, Georgia Institute of Technology, Ohio State University, Princeton University, Stanford University, Texas A&M University, the University of Chicago, the University of Utah, and the University of California, Davis.
Frontera will serve as powerful instrument for the largest and most computationally demanding users in the U.S., and a training ground for the next-generation of scientists.
"Academic researchers have never had a tool this powerful to solve the problems that matter to them", stated Dan Stanzione, TACC executive director. "We are proud to launch Frontera - a new, national resource for science that will power the discoveries of the future."