Today most high-end HPC systems employ data storage separate from the main computer system. As such, the I/O subsystem often struggles to deal with the degree of parallelism present simply because too many processors are trying to communicate with slow disk storage at once. As we move into the domain of extreme parallelism at the Exascale we need to address this I/O challenge if such systems are to deliver appropriate performance and efficiency for their application user communities.
Peter Bauer, head of ECMWF's Scalability Programme, stated: "Numerical weather prediction works on very tight schedules to produce forecast model runs and to manage the vast amounts of data involved in individual tasks, post-production and dissemination to users. The solutions that NEXTGenIO offers will greatly enhance efficiency by implementing a much closer link between computing and data management with novel technology. This will enable ECMWF Member and Co-operating States to benefit from enhanced model configurations without excessive data handling costs."
The NEXTGenIO project will explore the use of non-volatile memory technologies and associated systemware developments through a co-design process with three end-user partners: a high-end academic HPC service provider, a global numerical weather centre and a commercial on-demand HPC service provider. A key output of the project will be a prototype system built around NV-DIMMs by Intel and Fujitsu.
Karl Solchenbach, Director of Intel's data centre pathfinding activities in Europe, stated: "Through our collaboration with the world-class European HPC organisations on the NEXTGenIO project, we are able to advance the parallel I/O field and demonstrate the benefits of the breakthrough 3D XPoint technology, a new type of non-volatile memory."
"We are very proud to be part of the NEXTGenIO project", added Ingolf Stärk, Sr. Director HPC from Fujitsu. "Building the prototype system in an intensive co-design approach with other hardware, middleware and end user partners will make sure we can reach the improvements in performance and efficiency really up to the application layer. Bringing the benefits of the new memory technologies straight to end users through integrated HPC solutions is a key goal for Fujitsu in this project."
The partners will also develop an I/O workload simulator to allow quantitative improvements in I/O performance to be directly measured on the new system in a variety of research configurations. Systemware developed in the project will include performance analysis tools, improved job schedulers that take into account data locality and energy efficiency, optimised programming models, and APIs and drivers for optimal use of the new I/O hierarchy.
Professor Mark Parsons, Executive Director of EPCC and NEXTGenIO Project Coordinator, stated: "Intel's 3D XPoint memory is going to impact all areas of computing over the next 5 years. I am immensely proud that EPCC will lead its development in the HPC arena through the NEXTGenIO project. We have brought together an extremely strong partnership, all of whom are firmly committed to exploring this fascinating, transformational technology."
NEXTGenIO started on the 1st October 2015 and is set to run for 3 years. The consortium partners are EPCC, Intel, Fujitsu, Technische Universität Dresden, Barcelona Supercomputing Center, the European Centre for Medium-Range Weather Forecasts, Allinea and Arctur.
For more detailed information on the project you can visit www.nextgenio.eu .