"Overall, DNA sequencers in the life sciences are able to generate a terabyte - or one trillion bytes - of data a minute. This accumulation means the size of DNA sequence databases will increase 10-fold every 18 months", stated Wu Feng of the Department of Computer Science in the College of Engineering at Virginia Tech.
"In contrast, Moore's Law - named after Intel co-founder Gordon E. Moore - implies that a processor's capability to compute on such Big Data increases by only two-fold every 24 months. Clearly, the rate at which data is being generated is far outstripping a processor's capability to compute on it. Hence the need exists for accessible large-scale computing with multiple processors, though the rate at which the number of processors needs to increase is doing so at an exponential rate", Wu Feng added.
For the past two years, Wu Feng has led a research team that has now created a new generation of efficient data management and analysis software for large-scale, data-intensive scientific applications in the Cloud. Cloud computing is a term coined by computing geeks that in general describes a large number of connected computers located all over the world that can simultaneously run a programme at a large scale. Wu Feng announced his work in October at the O'Reilly Strata Conference + Hadoop World in New York City.
By background to Wu Feng's announcement, one needs to go back more than three years. In April of 2010, the National Science Foundation teamed with Microsoft on a collaborative Cloud computing agreement. One year later, they decided to fund 13 research projects to help researchers quickly integrate Cloud technology into their research.
Wu Feng was selected to lead one of these teams. His target was to develop an on-demand, Cloud-computing model, using the Microsoft Azure Cloud. It then evolved naturally to make use of the Microsoft's Hadoop-based Azure HDInsight Service. "Our goal was to keep up with the data deluge in the DNA sequencing space. Our result is that we are now analysing data faster, and we are also analysing it more intelligently", Wu Feng stated.
With this analysis, and the ability of researchers from all over the globe to see the same sets of data, collaborative work is facilitated on a 24/7 global perspective. "This co-operative Cloud computing solution allows life scientists and their institutions easy sharing of public data sets and helps facilitate large-scale collaborative research", Wu Feng added.
Think of the advantages of oncologists from Sloan Kettering to the German Cancer Research Center would have by maintaining simultaneous and instantaneous access to each other's data.
Specifically, Wu Feng and his team, Nabeel Mohamed, a master's student from Chennai, Tamilnadu, India and Heshan Lin, a research scientist in Virginia Tech's Department of Computer Science, developed two software-based research artifacts: SeqInCloud and CloudFlow. They are members of the Synergy Lab, directed by Wu Feng.
The first, an abbreviation for the words "sequencing in the Clouds", combined with the Microsoft Cloud computing platform and infrastructure, provides a portable Cloud solution for next-generation sequence analysis. This resource optimizes data management, such as data partitioning and data transfer, to deliver better performance and resource use of Cloud resources.
The second artifact, CloudFlow, is his team's scaffolding for managing work flows, such as SeqInCloud. A researcher can install this software to "allow the construction of pipelines that simultaneously use the client and the Cloud resources for running the pipeline and automating data transfers", Wu Feng stated.
"If this DNA data and associated resources are not shared, then life scientists and their institutions need to find the millions of dollars to establish and/or maintain their own supercomputing centres", Wu Feng added.
Wu Feng knows about high-performance computing. In 2011, he was the main architect of a supercomputer called HokieSpeed.
That year, HokieSpeed settled in at no. 96 on the Top500 List, the industry-standard ranking of the world's 500 fastest supercomputers. Its fame, however, came because of the machine's energy efficiency, recorded as the highest-ranked commodity supercomputer in the United States in 2011 on the Green500 List, a compilation of supercomputers that excel at using less energy to do more.
Economics was also key in Wu Feng's supercomputing success. HokieSpeed was built for $1.4 million, a small fraction - one-tenth of a percent of the cost - of the Top500's no. 1 supercomputer at the time, the K Computer from Japan. The majority of funding for HokieSpeed came from a $2 million National Science Foundation Major Research Instrumentation grant.
Wu Feng has also been working in the biotechnology arena for quite some time. One of his key awards was the NVIDIA Foundation's first worldwide research award for computing the cure for cancer. This grant, also awarded in 2011, enabled Wu Feng, the principal investigator, and his colleagues to create a client-based framework for faster genome analysis to make it easier for genomics researchers to identify mutations that are relevant to cancer. Likewise, the more general SeqInCloud and CloudFlow artifacts seek to achieve the same type of advances and more, but via a Cloud-based framework.
More recently, he is a member of a team that secured a $2 million grant from the National Science Foundation and the National Institutes of Health to develop core techniques that would enable researchers to innovatively leverage high-performance computing to analyse the data deluge of high-throughput DNA sequencing, also known as next-generation sequencing.