Expansion in Imperial College London's Research Computing Services over the past decade had resulted in over 30 separate and independently managed islands of storage - silos that were difficult to access, manage and use. The group needed to enable researchers to access data with ease and speed, so they remain solely focused on their research projects - while integrating legacy and future compute systems and managing data throughout its life-cycle in line with stringent regulatory compliance guidelines.
The new Research Data Store (RDS) is built around ArcaStream PixStor, a high-performance scalable storage platform based on IBM Spectrum Scale parallel file system, which combines flash, disk, tape, and Cloud storage into a single global name space. The infrastructure is geographically dispersed with a 5PB storage repository at a primary site and a secondary site for disaster recovery, served by PixStor with asynchronous replication and intelligent tiering to external storage targets. RDS also includes Excelero's NVMesh, software that enables the sharing of NVMe Flash storage resources across any network and supports any local or distributed file system. NVMesh provides a scalable NVMe tier for extreme metadata performance. Users benefit from the performance of local flash with the convenience of centralized storage while reducing the overall storage total cost of ownership.
With this new robust infrastructure, Imperial College London's RDS now simultaneously serves 2000 existing HPC nodes and over 3000 users seamlessly, with 20GB/s of throughput with no loss of interactive user performance.
"The usability of the systems for interactive use in particular has improved significantly", explained Matthew Harvey, RDS project lead and RCS Manager at Imperial College. "Previously, there were frequent interruptions to interactive use because the file system load for some compute jobs effectively squeezed out users. Users would log into the system, type in their search criteria but it could take more than 10 seconds to respond. Now that is a thing of the past."
The new RDS supports a charge-back strategy where researchers cost-out storage as services on their grants - instead of charging users based on reserve capacity. More effective management of storage capacity also allowed the College to avoid costly additions. The ArcaStream platform provides tools and insight needed to understand the access patterns of data on the file system for each project allocation. "This information governance is enabling us to store valuable data more intelligently and economically", Imperial College London's Matthew Harvey continued.
Excelero and ArcaStream will showcase the performance and efficiency of their solutions at the ISC High Performance Event 2019, June 16-20 in Frankfurt in Excelero's booth E-1039.
Cray to deliver first exabyte storage system for the Frontier exascale system at ORNL ...
NVIDIA brings CUDA to Arm, enabling new path to Exascale supercomputing ...
Quantum software company Zapata Computing adds Rhonda Germany Ballintyn to Board ...
IBM expands its quantum computing programme to Africa with Wits University ...
Kalray at the heart of the European Processor Initiative ...
Extension CERN Alice and LHCb-data centres: 50% delivered ...
The Luxembourgian supercomputer "Meluxina" will be part of the European network EuroHPC ...
Joint PRACE IAC and EuroHPC INFRAG Workshop on Support for Industrial Users ...
ISC 2019 kicks off with five days of non-stop HPC ...
Italy to host one of the supercomputers funded by EuroHPC ...
Bright Computing and Cray reduce complexity and cost in modern HPC environment for NeSI and NIWA ...
Panasas welcomes Boston Limited to its accelerate Channel Partner Programme ...
Cray powers Geospatial AI revolution with breakthrough deep learning performance ...
E4 Computer Engineering and the Open Edge and HPC Initiative announce the codes@OEHI programme ...
WekaIO updates file system with advanced security features for multi-user enterprise HPC ...
Student attending OSC's Summer Institute will solve problems with supercomputers ...
Immortal quantum particles ...
Energy department to invest $32 million in computer design of materials ...
TOP500 becomes a Petaflop club for supercomputers ...