How do you think Cloud computing for HPC has progressed over, say, the last five years? And what developments have driven this evolution?
Stowe:The past five years have been a period of tremendous growth for HPC in the Cloud or utility HPC - it's clearly reached the point of global awareness and mainstream adoption. Individual users and companies as a whole have realized that on-demand access to the right compute and data resources at the right time makes better science and better business possible faster and at less cost than possible before. A number of developments have influenced this growth. Global pressures to accelerate the pace of innovation, an insatiable demand for compute and data, and an economic climate of austerity and budget constraints have created a perfect storm that's fueled the need for utility HPC.
What important barriers remain to be overcome?
Stowe:Security continues to be an important consideration. Having your own local set of encryption keys, that your Cloud provider does not have, can be critical for ensuring that your data is protected. Doing proper reviews, and enabling proper audit trails for utility HPC systems is also important. Any infrastructure provider that has multiple clients interacting in the same operating system instance on a host is a risk, but thankfully the large Cloud providers all handle this properly. Also, as organisations leverage utility HPC more and more, centrally orchestrating data movement and access from lower cost "cold storage" as applications demand it will be a key requirement. One top 10 pharma company uses Cycle's Data Manager to automate archival and retrieval of 75-terabyte data sets to and from Amazon Glacier, without the need to re-code or disrupt their application's work flow.
Many use cases for Cloud-enabled technical computing seem to be in the life science realm. What do you attribute this to?
Stowe:Many of life science workloads, such as genome sequencing, or needle-in-a-haystack simulations like drug design are "pleasantly parallel" or high throughput, where computations are independent of each other. In the case of drug design, a cancer target is a protein that, much like a lock, has a pocket where molecules can fit, like keys, to either enhance or inhibit its function. The problem is, rather than the tens of keys on a normal key chain, you have tens of millions of molecules to check. Each one is computationally intensive to simulate, so in this case, a drug designer has approximately 340,000 hours of computation, or nearly 40 compute years, ahead of herself. With utility HPC, what would have taken a year to set up at a price-tag of $44 million, this drug sequencing workload completed in just 11 hours at a cost of $4,372. Without utility HPC, it's safe to say this science would never happen. Even though life science was a logical proving ground for HPC in the beginning, other industries - financial services/insurance, EDA, manufacturing, and even energy, are now capitalizing on these kinds of benefits.
What other types of HPC applications or industries do you think are most suitable for the utility model of computing at this point?
Stowe:We think that utility HPC will be the single largest accelerator of human invention in the coming decades. We have many use cases - energy, manufacturing, financial services, and many more - that prove how most modern sciences, especially Monte Carlo or data parallel simulations, work great in the Cloud. Researchers, quants, and scientists of all disciplines can now execute computational science and complex finer-grained analysis that was previously unapproachable due to cost or overhead. Consider the impact on Financial Services as an example: a Fortune 100 firm uses HPC in the Cloud to launch its monthly risk report - a 2.5 million compute hour Monte Carlo simulation that now completes over a weekend. A Fortune 1000 life insurance firm dynamically hedges risk across its entire portfolio, with nested stochastic on stochastic models and billions of inner paths for each annuity. Even at smaller scales, where scientists can start work in 10 minutes instead of waiting 6 weeks to get five servers, great science can now be done in a wide range of industries and applications.