RSC demonstrated on its booth at ISC'14 the RSC PetaStream system: a revolutionary ultra-high density solution with massively parallel architecture, direct liquid cooling and the newest Intel Xeon Phi 7120D setting one more world record of computing density of 1.2 PFLOPS peak performance per cabinet within just 1 m2, or 11 ft2, footprint. RSC PetaStream became even 20% more powerful since its first announcement with the previous world's record of 1 PFLOPS per rack at SC'13 in Denver, USA. The RSC's new technology breakthrough was achieved with the newest Intel Xeon Phi 7120D coprocessor launched in March, 2014. RSC PetaStream became the world's first HPC solution with Intel Xeon Phi 7120D.
One RSC PetaStream cabinet contains 1024 interchangeable computing nodes with 250,000 execution threads and using RSC's direct liquid cooling technology allows efficiently removing up to 400 kW of heat. This value is 4x times greater than the previous RSC Tornado architecture achievement and sets the new world record of power density.
"We were the first in the world to develop and introduce to users ultra high dense massively parallel system based on the newest Intel Xeon Phi 7120D. RSC PetaStream's leading characteristics were achieved as a result of the hard work and extensive experience of RSC engineers in development of extra efficient direct liquid cooling technologies and ultra-high density integration of supercomputers based on standard server components and technologies. It helped us to achieve such a breakthrough setting the new world records of computing density with 1.2 PFLOPS peak performance and power density of 400 kW per cabinet within just 1 m2, or 11 ft2, footprint", stated Alexey Shmelev, COO of RSC Group.
RSC PetaStream solution is an innovative implementation of massively parallel architecture using the best available industrial components, highly dense integration and unique system architecture delivered by the RSC engineers. Every compute node is independent and interchangeable, based on Intel Xeon Phi 7120D coprocessor featuring 61 cores and 16GB of high-bandwidth GDDR5 memory and works under Linux family operating system's control. All nodes of the RSC PetaStream system are tightly interconnected using high speed InfiniBand FDR network.
Thanks to the x86 architecture of the chosen microprocessor, the developers can use existing programming models and run existing scientific applications on the ExaScale-class supercomputer. At the same time, the flexibility of the RSC PetaStream architecture allows to develop, optimize and test new innovative programming models and applications for the future massively parallel supercomputers based on the perspective of many-core processors.
In addition to the highest performance provided by the RSC PetaStream, the use of highly efficient power distribution based on 400V direct current and RSC's proven direct liquid cooling technology enabled work record computational density of 1.2 PFLOPS in one rack with just 2.2 m3 or 77 ft3 volume. High energy efficiency and scalability leads to over 2x reduction of the electricity usage comparing to traditional HPC cluster systems when running applications with high degree of parallelism.
The new technologies developed by RSC for the RSC PetaStream are the significant step towards practical development of ExaScale-level supercomputers, providing millions of execution threads for the user applications.
Using the RSC PetaStream in practice has already enabled a number of promising scientific discoveries. Some of the research results were presented at the International Supercomputer Conference ISC'14 in Leipzig, Germany.
The AstroPhi astrophysics code developed by research scientists of the Institute of Computational Mathematics and Mathematical Geophysics of the Siberian Branch of the Russian Academy of Sciences (ICMMG SB RAS) and Novosibirsk State University (NSU) was used on the RSC PetaStream system to study collisions of galaxies.
"Collision of galaxies is an ordinary event in their evolution. Every galaxy commits up to ten collisions per Hubble time and the collisions lead to their wide diversity of galaxies observed. Supercomputer simulation of such processes is the only way to study them. With the increasing number of supercomputers with hybrid architecture the most urgent problem is the creation of the efficient numerical codes for such computing platforms. It cannot be done without co-design of the entire computing platform, its software and applications. Thanks to the innovative technology developed by RSC Group and the opportunity to test the AstroPhi application on the RSC PetaStream with Intel Xeon Phi based nodes, the simulation of the collision of galaxies can be done with the record resolution. The calculation speed on a single RSC PetaStream module is 6x higher than on 4 nodes based on Intel Xeon E5-2690 processors", noted Igor Chernykh from ICMMG SB RAS and Igor Kulikov from NSU.
Researchers from St. Petersburg State Polytechnic University (SPbSPU) and Petersburg Nuclear Physics Institute NRC "Kurchatov Institute" (PNPI NRC KI) held series of simulations and obtained new results with the RSC PetaStream HPC system. Study of processes vital for the cell lifecycle, aging, and tumour genesis requires research of the fundamental laws and regulation mechanisms of gene transcription - the reading process of genetic information.
The aim of this work is to study various intermediate assembly states of nucleosomes using molecular dynamics (MD) simulation methods - one of the results of these methods are particles trajectories - available in the open source software package GROMACS. Solving such problems requires significant computing resources - the typical time-step size is 1-2 fs, with the meaningful observation time ranges around 100 ns. The simulated system contains a large number of point charges distributed in the volume that usually has a negative impact on the scalability and can be solved by adjusting of the MD-engine parameters.
"GROMACS MD-engine has optimizations for different hardware architectures that enable effective application on many supercomputers, with scalability up to the level of 200 atoms per core. The latest versions of GROMACS have support for native execution on the Intel Xeon Phi coprocessor that allows us to use the RSC PetaStream system. Even using a single RSC PetaStream computing module we achieved 4.6 TFLOPS level of sustained performance and solution rate of 1,9 ns/day, which is comparable to 10 dual-processor nodes with the Intel Xeon E5-2695 v2. Using massively parallel systems like the RSC PetaStream makes it possible to study broader time scales - up to milliseconds - and larger system", explained Evgeniy Petukhov, Head of System Software Department, SPbSTU and Alexey Shvetsov, Junior Scientist, Department of Molecular and Radiation Biophysics PNPI NRC KI.
The research, intended for solving an important epidemiological problem - the tracking of the dynamics of the spread of infection during the epidemic in Finland 2009-2011 - is being conducted by an international team of researchers from the University of Helsinki, National Institute of Health and Welfare in Finland, and Rybinsk State Aviation Technical University in Russia. Researchers propose a Bayesian model of the spread of infection and corresponding implementation based on the importance sampling approach. The main purpose is to estimate the number of hidden (unobserved) cases of illness to effectively design control measures, such as vaccinations or quarantines. The computational statistics methods based on the Monte-Carlo approach used for the estimation require significant computational resources to get the values of likelihood function for given set of parameters - the more iterations performed, the better the estimation is.
"One RSC PetaStream module based on massively parallel architecture with eight Intel Xeon Phi provides the required performance level for MCMC-sampling method to obtain optimal trajectories and enabled a simulation speed of more than 360,000 Monte-Carlo casts per second. Such high performance gives the possibility to make precise estimations of unobserved cases of illness to effectively track spread of the infection and prevent pandemics", stated the team of researchers.
At the Chemistry department of the Lomonosov Moscow State University with support from the Innovative Computing Laboratory of the University of Tennessee, the MAGMA linear algebra library has been adopted to the new hardware platform and its performance explored. It has been shown that achieved performance can be more that 60% of theoretical (peak) performance that additionally re-instates the computational efficiency of the system. The results are published as a part of ISC'14 scientific programme.
RSC specialists have analyzed the long-term experience of development, deployment and operation of high-performance cluster systems for HPC and data centre segments and optimized the approach to creating solutions based on the time proven RSC Tornado cluster architecture in the area of implementing computing system management and the building of server node configurations. This enabled transition to the base configuration of the RSC Tornado node - a 2-processor configuration based on high-performance Intel Xeon E5-2600 v2 processors - and additional expansion packs that most fully meet specific requirements of various customers.
For example, the RSC Tornado HPC Expansion Pack focused on high-performance computing with two Intel Xeon Phi coprocessors improves computing node performance to 2.93 TFLOPS.
RSC Tornado BigData Expansion Pack provides price-performance optimized configuration for high-performance data processing.
RSC Tornado VDI Expansion Pack is intended for virtualization and remote workplace access and for CAD/CAM/CAE fields. It is based on AMD FirePro S10000 or NVIDIA GRID K1/K2 cards and on Intel SSD DC P3700 high-performance solid-state drives.
RSC Tornado Security&Protection Expansion Pack is developed specifically for customers from the financial vertical market and other users that require maximum access speed and data protection.
Hardware and software package RSC SDM Expansion Pack implements a Software Defined Management concept developed by RSC specialists for RSC Tornado and RSC PetaStream nodes improving manageability and accessibility of computing systems for user applications. Flexible combination of management agents - infrastructure components, computing nodes, scheduler and user application libraries - within the scope of this concept enables the creation of new Task-Oriented Management Applications (TOMA) that improve computing system efficiency and reduce operating costs. One of the examples of this is an application for adaptive management of task scheduler priorities that considers available power resources and actual capacity of data centre cooling systems to prevent task termination and resulting loss of critical data on temporary infrastructure failures.
Other specialized expansion packs can also be created on specific requests of the customers. This approach has significantly improved availability of specialized configurations and made the modernization process much more easy while preserving record compactness, computing density and power density of solutions based on the RSC Tornado cluster architecture.