As one of the top five scientific institutions in the world specializing in DNA sequencing, Sanger Institute embraces the latest technologies to research the genetic basis of global health problems, including cancer, malaria, diabetes, obesity and infectious diseases.
In order to manage the massive surge in the volume of data required to evaluate genetic sequences, Sanger Institute selected DDN's SFA high-performance storage engine and EXAScaler Lustre file system appliance to deliver unprecedented levels of throughput and scalability to support tens of thousands of data sequences requiring up to 10,000 CPU hours of computational analysis.
With more than 2,000 scientists around the world, DDN SFA storage will also help facilitate data access and sharing including for those who access data through the Sanger Institute's website, which results in 20 million hits and 12 million impressions each week.
As the 30 DNA sequencers in Sanger Institute's Illumina Production Sequencing core facility each pump out about one terabyte of data daily, with DDN technology the Sanger Institute has an easy-to-manage, integrated system that offers unparalleled scalability to address both complex computing problems and ever-changing collaboration requirements associated with its leading-edge research.
DDN's proven experience serving some of the world's fastest computers ensures that the Sanger Institute can deliver the highest levels of compute performance and throughput, as well as maximum system uptime, to optimize the latest sequencing technologies. This is critical as today's sequencers produce a million times more data than those used a decade ago.
Moreover, the institute now can provide its diverse scientific community with an essential tool for leveraging its approximately £80 million research budget to the fullest in order to further the exploration of groundbreaking scientific and medical discoveries.
With DDN storage, the Sanger Institute can achieve its goal of supporting different research workloads with a wide range of computational analysis and storage requirements while being able to expand quickly and without disruption.
Since installing its initial SFA storage platform, the Sanger Institute keeps pace with ever-increasing computational and analytical demands by taking advantage of DDN's ongoing performance increases to achieve speeds of up to 20 GBps, which enables meeting the needs of the most demanding workloads.
To accommodate demands for increased bandwidth, Sanger Institute is upgrading its 10GbE network to 40GbE and plans to scale its current DDN storage to support expanded network capacity.
Additionally, Sanger Institute is exploring DDN WOS distributed object storage platform, which could be ideal for increased collaboration and data sharing as part of a private Cloud.
Tim Cutts, acting head of scientific computing, Wellcome Trust Sanger Institute, stated: "If you need 10,000 cores to perform an extra layer of analysis in an hour, you have to scale a significant cluster to get answers quickly. You need a real solution that can address everything from very small to extremely large data sets. We have to explore emerging technologies that could play a significant role in our future architecture. We need solutions that give us a much better way to provide storage to our expanding user community with good access controls through iRODS."
Phil Butcher, director of information communications technology, Wellcome Trust Sanger Institute, stated: "The sequencing machines that run today produce a million times more data than the machine used in the human genome project. We produce more sequences in one hour than we did in our first 10 years. For instance, a single cancer genome project sequences data that requires up to 10,000 CPU hours for analysis and we're doing tens of thousands of these at once. The sheer scale is enormous and the computational effort required is huge. Our storage strategy gives us incredible scaling. If we need to add a new sequencer, we can expand quickly and without disruption."