Tiago Quintino explained in his presentation that ECMWF runs several simulation models that run weather forecasts on their supercomputer. Simulations are run every day, so there is a time critical component. The quality of the model can be improved by doing calculations on smaller "grids". Currently the smallest grid is 9 km by 9km. To get better weather predictions the grid size must be reduced. Also the number of different related simulations, called ensembles, should be increased. This leads to much more data that needs to be processed. If you do not want to increase the predication time, you need a faster computer, hence ECMWF's look into exascale supercomputing.
Currently, getting data in and out of the computer's processors fast enough is a bottleneck. This is called the I/O issue. Disks are slow. Flash disks are faster than rotating disks, but still slow compared to memory speed. A typical next generation forecast on a 1,25 km grid containing a 50 times increase of ensemble size, would lead to about 3,6 petabyte of data to be handled. This would need an exascale computer to process.
This is where the NEXTGenIO project comes in. NEXTGenIO produced a prototype HPC system based on Intel 3D XPoint technology. ECMWF's task in the project was to explore the interaction within the I/O server layer and to test and assess the system scalability. NEXTGenIO was a co-design project, so ECMWF also brought in application expertise and interacted with hardware, middleware and software developers.
The experiments using NVRAM on the NEXTGenIO system are very promising. It removed the bottleneck from reading and writing to the parallel file system on disk. And for realistic experiments, it is not even used up to its full speed: it could handle more data.
The current NEXTGenIO prototype which is located at EPCC in Edinburgh contains 16 nodes. A system containing 213 of these nodes would be able to handle the full 1,25 km forecast. So, from an I/O point of view, NEXTGenIO showed it is ready for exascale. ECMWF expects to run these models by 2035, by then the technology will be further improved and cheaper than today.
But using this type of NVRAM could also open up new applications.
Watch the full presentation of Tiago Quintino: