Back to Table of contents

Primeur weekly 2015-11-16

Focus

eScience and open access are really taking off now globally, according to Tony Hey ...

Exascale supercomputing

NEXTGenIO project tackles HPC's I/O challenges ...

EXDCI: towards a common HPC strategy in Europe ...

TERA 1000: CEA completes first milestone towards Exascale ...

National Labs collaborate to shape development of next-generation supercomputers ...

Quantum computing

Los Alamos National Laboratory orders a 1000+ Qubit D-Wave 2X quantum computer ...

Focus on Europe

DDN Storage awarded 18+ petabyte hyper-converged storage contract at CEA major French government research site ...

ARM accelerates mathematical computation on 64-bit ARM-based HPC systems ...

Kitware and NICE collaborate to push visualization to the next level ...

The Alan Turing Institute and Intel to form strategic partnership ...

Eurotech HiVe HPC system now generally available with ARM CPUs ...

Middleware

Somerville release of Einstein toolkit launched ...

New Cactus Computational Toolkit release launched ...

Bright Computing announces integration with Dell PowerEdge servers for HPC environments ...

High performance computing leaders unite to develop open source framework ...

Altair to open source PBS Professional HPC technology in 2016 ...

Hardware

Altera FPGAs accelerate servers at Texas Advanced Computing Center ...

DDN introduces the world's fastest hybrid SSD and disk, high density embedded storage platform ...

DDN storage selected by Texas Advanced Computing Center (TACC) to deliver a 40x speed up in scientific research and innovation ...

DDN Annual High Performance Computing Trends survey reveals that data storage has become the most strategic part of the HPC data centre ...

Supercomputer project to study solid-state drives under data-intensive workloads ...

Cray to join OpenHPC Community ...

PEZY and Imagination team up to develop next-generation HPC systems ...

Applications

Intersect360 Research issues Processors Report as part of the HPC User Site Census series ...

Oak Ridge National Laboratory and industry partners use high-performance computing to drive alloy design ...

Berkeley Lab-UC Berkeley team's programme can reconstruct entire genomes in just minutes ...

HPC Advisory Council announces Stanford High-Performance Computing Conference 2016 ...

Boston Children's Hospital to tap IBM Watson to tackle rare paediatric diseases ...

TOP500

China's Tianhe-2 supercomputer maintains top spot on list of world's TOP500 supercomputers ...

The Cloud

NUMECA steps farther into the era of software as a service with UberCloud containers ...

Markets and Markets issues report on Cloud HPC market ...

DDN Annual High Performance Computing Trends survey reveals that data storage has become the most strategic part of the HPC data centre


12 Nov 2015 Santa Clara - DataDirect Networks has issued the results of its annual High Performance Computing (HPC) Trends survey, which showed that HPC end-users view data storage as the most strategic part of the HPC data centre and that managing mixed I/O performance and rapid data growth remain the biggest challenges for HPC organisations going into 2016.

Conducted by DDN for the third consecutive year, the survey polled a cross-section of end-users managing data intensive infrastructures worldwide representing hundreds of petabytes of storage investment. Respondents included individuals responsible for high performance compute, networking and storage systems from Financial Services, Government, Higher Education, Life Sciences, Manufacturing, National Labs and Oil & Gas organisations. The data under management in each of these organisations is staggering.

Of organisations surveyed:

  • 68 percent manage or use more than one petabyte of data storage; and
  • More than 25 percent manage or use more than 10 PBs of data storage, which represents a 69 percent increase versus 2014 survey results.

Storage is the fastest growing segment in IT spending in HPC, and according to an overwhelming majority of survey respondents (77 percent) data and data storage has now become the most strategic part of the HPC data centre as end-users seek to solve data access, workflow and analytics challenges to accelerate time to results.

The diverse set of applications driving this ever growing data in large scale environment and analytical workflows places rigorous demands on storage infrastructures and creates unique challenges for HPC users.

As the graph highlights:

  • Performance ranks as the number one storage and Big Data challenge by approximately two-thirds (66 percent) of those polled; and
  • Mixed I/O performance was cited as the biggest concern by more than half (53 percent) of the respondents, which represented an eight percentage point increase compared with last year's survey results.

More than half of respondents (56 percent) identify storage I/O as the main bottleneck in analytics workflows.

These responses illustrate how performance issues have surged as Big Data environments grapple with a proliferation of diverse applications creating mixed IO patterns and stifling the performance of their storage infrastructure.

Only a small percent of respondents believe today's file systems and data management technologies will be able to scale to Exascale levels, while almost 2/3 of respondents believe new innovation will be required. This belief is exemplified in respondents’ views on addressing performance issues:

  • 58 percent view burst buffers to be the most likely technology to push storage to the next level as users seek faster and more efficient ways to offload I/O from compute resources, to separate storage bandwidth acquisition from capacity acquisition and to support parallel file systems to meet Exascale requirements.

Given the intense performance requirements of HPC environments as well as security, data access, workflow management and cost considerations, it is not surprising that according to 75 percent of those surveyed, security and data sharing complexity remain the largest impediments to increased collaboration across sites. This concern is underscored by respondents’ overwhelming preference for private Clouds:

  • By a ratio of 3:1 private Clouds were preferred over public clouds for HPC users' existing Cloud deployments.

With storage performance a critical requirement for today's large scale and petascale-level data centres, site-wide file systems like those at Oakridge National Lab (ORNL), National Energy Research Scientific Computing Center (NERSC) and Texas Advanced Computing Center (TACC) continue to be a significant infrastructure trend in HPC environments, according to more than two-thirds (67 percent) of HPC customers polled. Site-wide file systems allow architects to either consolidate multiple computers on the same system and/or provide the flexibility to upgrade storage and servers as needed.

"The results of DDN’s annual HPC Trends Survey reaffirms the key issues and requisite requirements we hear from HPC users regularly. Handling mixed I/O workloads and resolving I/O bottlenecks in data intensive workloads are critical issues DDN has worked fervently to address within our suite of high performance end-to-end solutions", stated Molly Rector, CMO, executive vice president product management and worldwide marketing, DDN. "Survey respondents were clear that data storage has become the most strategic part of the data centre, and users rely on DDN to deliver continued technology innovation that enables research and business organisations to accelerate strategic insights and to speed time to results. DDN remains committed to the relentless innovation that raises the bar to meet the surging I/O performance, scale, content distribution and collaboration demands of high performance computing - in ways that are faster, smarter, more efficient and more cost effective than anyone else in the market."
Source: DataDirect Networks

Back to Table of contents

Primeur weekly 2015-11-16

Focus

eScience and open access are really taking off now globally, according to Tony Hey ...

Exascale supercomputing

NEXTGenIO project tackles HPC's I/O challenges ...

EXDCI: towards a common HPC strategy in Europe ...

TERA 1000: CEA completes first milestone towards Exascale ...

National Labs collaborate to shape development of next-generation supercomputers ...

Quantum computing

Los Alamos National Laboratory orders a 1000+ Qubit D-Wave 2X quantum computer ...

Focus on Europe

DDN Storage awarded 18+ petabyte hyper-converged storage contract at CEA major French government research site ...

ARM accelerates mathematical computation on 64-bit ARM-based HPC systems ...

Kitware and NICE collaborate to push visualization to the next level ...

The Alan Turing Institute and Intel to form strategic partnership ...

Eurotech HiVe HPC system now generally available with ARM CPUs ...

Middleware

Somerville release of Einstein toolkit launched ...

New Cactus Computational Toolkit release launched ...

Bright Computing announces integration with Dell PowerEdge servers for HPC environments ...

High performance computing leaders unite to develop open source framework ...

Altair to open source PBS Professional HPC technology in 2016 ...

Hardware

Altera FPGAs accelerate servers at Texas Advanced Computing Center ...

DDN introduces the world's fastest hybrid SSD and disk, high density embedded storage platform ...

DDN storage selected by Texas Advanced Computing Center (TACC) to deliver a 40x speed up in scientific research and innovation ...

DDN Annual High Performance Computing Trends survey reveals that data storage has become the most strategic part of the HPC data centre ...

Supercomputer project to study solid-state drives under data-intensive workloads ...

Cray to join OpenHPC Community ...

PEZY and Imagination team up to develop next-generation HPC systems ...

Applications

Intersect360 Research issues Processors Report as part of the HPC User Site Census series ...

Oak Ridge National Laboratory and industry partners use high-performance computing to drive alloy design ...

Berkeley Lab-UC Berkeley team's programme can reconstruct entire genomes in just minutes ...

HPC Advisory Council announces Stanford High-Performance Computing Conference 2016 ...

Boston Children's Hospital to tap IBM Watson to tackle rare paediatric diseases ...

TOP500

China's Tianhe-2 supercomputer maintains top spot on list of world's TOP500 supercomputers ...

The Cloud

NUMECA steps farther into the era of software as a service with UberCloud containers ...

Markets and Markets issues report on Cloud HPC market ...