SuperMUC system at Garching proves to be a stable workhorse

25 Jun 2014 Leipzig - In the session "HPC in Europe" at ISC'14 in Leipzig, Arndt Bode from the Leibniz Supercomputing Center and the Technical University in Munich, talked about the experiences in the realm of power efficiency with the SuperMUC system in Garching. He stated that the energy efficient infrastructure is still under development but is showing good progress. Further evolution in the quality of applications will bring down the total cost of ownership (TCO) for the SuperMUC system.

A better use of the hardware of the SuperMUC system is required to have a better Total Cost of Operation (TCO). The first experiences show a 5-fold improvement for the SuperMUC, stated Arndt Bode. To become more energy-efficient is the second option.

There has been a recent configuration update for the SuperMUC. SuperMUC is located at the campus in Garching. With its 3.2 PFlop/s, the machine ranks now at no. 12 in the TOP500 list. There will be an upgrade to 6.4 PetaFlop/s so Moore's Law still holds, Arndt Bode smiled.

The SuperMUC is an IBM i Dataplex dx360 M4 system and is watercooled.

The interconnect is InfiniBand. There was a migration system from SGI Altix to IBM iDataPlex.

In addition, SuperMIC is a special prototype Intel Phi system with 128 nodes SB-EP and 2048 cores for the astrophysics community with 250TB of storage.

Arndt Bode expanded on the applications. At Garching, workshops are being organized such as the Results Workshop, scheduled for July 8-9, 2014. At this workshop, more than 110 application codes will be addressed in the areas of astrophysics, plasma physics, earth and environmental sicences, etc.

Extreme scale workshops are being organized on the SuperMUC@LRZ. Exclusive block operation time has been reserved for selected projects. During this reserved time, researchers will perform serveral actions including testing, debugging, performance tuning, and production, according to Arndt Bode.

The goal is to run applications on 18 islands, consisting of 144.000 cores. The first workshop was in July 2013. A pre-requisite for these workshops is the scaling plot up to 4 islands.

There have been some lessons learnt from the first workshop, Arndt Bode explained. Important elements are the physical attendance of participants; the one-to-one support during the workshop; the fine grained planning of the schedule; and the different day and night time operation modes.

Arndt Bode also expanded on the statistics including the energy per wallclock. Several power bands have been running.

The nightly runs included:

  • 3 runs with 18 islands
  • 5 runs with 16 islands
  • SeisSol: 4th production run
  • Gadget: 7th production run

There were 6 new full-system applications ande 4 already existing full-system applications.

In terms of future support for the users, the ExScaMIC parallel supercomputing centre plays a key role. The Intel Phi architecture is currently the most attractive hardware accelerator achiever, stated Arndt Bode.

The type of cooling contributes to the energy efficiency. The cooling is based on warm water but also on different types of chilled water.

The power usage efficiency amounts to 1.18 PUE.

The introduced average power consumption constraint during the maintenance of the cooling towers is also being measured. Can the job J be scheduled and the power consumotion constraint be preserved? This is an important question, according to Arndt Bode.

The Leibniz Supercomputing Center is using the Adaptive Energy and Power Consumption Prediction Process and Model tool to measure the power efficiency. The monitoring tool is the PowerDAM to predict estimation functions and the average power consumption.

Arndt Bode said that the energy efficient infrastructure is in good progress. The hardware and software tools for energy efficiency are under development. The quality of the applications is very important as well.

The SuperMUC is a large homogenous system with variations, explained Arndt Bode, but it has already proven to be a stable working horse.
Leslie Versweyveld