Two heterogeneous systems, based on NVIDIA's Kepler K20 GPU accelerators, claim the top two positions and break through the three-billion floating-point operations per second (gigaflops or GFLOPS) per watt barrier. Eurora, located at Cineca, debuts at the top of the Green500 at 3.21 gigaflops/watt, followed closely by Aurora Tigon at 3.18 gigaflops/watt. The energy efficiency of these machines, manufactured by Eurotech, improves upon the previous greenest supercomputer in the world by nearly 30%. Two other heterogeneous systems, Beacon with an efficiency of 2.449 gigaflops/watt and SANAM with an efficiency of 2.35 gigaflops/watt, come in at numbers 3 and 4 on the Green500. The former is based on Intel Xeon Phi 5110P coprocessors while the latter is based on AMD FirePro S10000 GPUs.
Rounding out the top five is CADMOS BlueGene/Q, which is based on a previously list-leading custom design of the IBM BlueGene/Q architecture with PowerPC-based CPU processors. In fact, the next swath of supercomputers down to no. 28 are dominated by IBM BlueGene/Q systems with one set at approximately 2.30 gigaflops/watt and another at approximately 2.18 gigaflops/watt. Overall, the greenest fifty systems are either heterogeneous systems, which incorporate accelerators (GPUs) or coprocessors, or custom BlueGene/Q systems. The exceptions are a pair of homogeneous systems at 40 and 41, which are the only homogeneous systems based on Intel's new Ivy Bridge processors and three more at 45-47 based on Intel Sandy Bridge processors.
The current fastest supercomputer in the world, Tianhe-2, uses heterogeneous computing elements based on Intel Xeon Phi. It delivers 1.9 gigaflops/watt, which is commensurate with the majority of Xeon Phi systems that are ranked between no. 30 and 35, inclusive.
With the DARPA's goal of an exascale supercomputer within a power envelope of 20 megawatts (MW), extrapolating Beacon, the previous no. 1 supercomputer on the Green500, to exascale results in a 408 MW machine was a challenge. However, due to the improved efficiency of Eurora, this exascale power envelope comes down to 312 MW, a sizeable 24% drop in power consumption for an exascale machine - a move in the right direction. Nevertheless, the electricity bills for such a system would still be more than 300 million U.S. dollars per year.
This 13th edition of the Green500 List also marks the Green500's first use of new energy measurement methodologies developed in tandem with the Energy Efficient High Performance Computing Working Group, Top500, and The Green Grid. In addition to the current Green500 submission requirements, denoted as Level 1, the Green500 also accepts higher-precision measurements, denoted as Level 2 and 3. The Green500 received Level 2 and 3 submissions from several systems, including a superset of the Sequoia BlueGene/Q supercomputer at LLNL called Sequoia-25 and SuperMUC from LRZ.
"While the higher quality measurements taken for these systems show lower energy efficiency, they provide a much better picture of the real-world costs of running each system as well as a more in-depth picture of how the system handles a Linpack run", noted Wu Feng, founder of the Green500. "Additionally, the Sequoia-25 Level 1 submission includes the networking infrastructure, as required for higher levels."
Since the launch of the Green500 in 2007, the energy efficiency of the highest-ranked machines on the Green500 has been improving much more rapidly than the mean and the median. For instance, while the energy efficiency of the greenest system improved by nearly 30%, the median improved by 14% and the mean by only 11%.
"Overall, the performance of machines on the Green500 List has increased at a higher rate than their power consumption. That's why the machines' efficiencies are going up", stated Wu Feng. For commodity-based machines - machines built with off-the-shelf components - a great deal of the efficiency gains can be attributed to heterogeneous designs, i.e., using traditional CPUs along with GPUs or coprocessors. Such a design allows these systems to keep pace and in some cases even outpace custom systems like IBM's Blue Gene/Q. "While the gains at the top end of the Green500 appear impressive, overall the improvements have been much more modest", stated Wu Feng. "This clearly indicates that there is still work to be done."
The Green500 has provided a ranking of the most energy-efficient supercomputers in the world since November 2007. For decades, the notion of supercomputer "performance" has been synonymous with "speed" - as measured in FLOPS, short for floating-point operations per second. This particular focus has led to the emergence of supercomputers that consume egregious amounts of electrical power and produce so much heat that extravagant cooling facilities must be constructed to ensure proper operation.
In addition, when there is an emphasis on speed as the ultimate metric, it often comes at the expense of other performance metrics, such as energy efficiency, reliability, availability, and usability. As a result, there has been an extraordinary increase in the total cost of ownership (TCO) of a supercomputer. Consequently, the Green500 seeks to raise the awareness in energy efficiency of supercomputing and to drive it as a first-order design constraint on par with speed.