Back to Table of contents

Primeur weekly 2016-08-01

Special

National Strategic Computing Initiative Executive Council to unfold Strategic Plan ...

Quantum information science is to play a key role in the U.S. National Strategic Computing Initiative ...

Focus

Lenovo is heavily investing in new HPC technology, showing its commitment to the HPC market ...

South Africa's CHPC in the picture with new Lengau supercomputer, Student Cluster Competition victory and upcoming ICRI 2016 Conference ...

Exascale supercomputing

China starts developing new-era exascale supercomputer ...

Crowd computing

IBM intensifies fight against Zika ...

Middleware

Red Hat Satellite 6.2 introduces new features to help users increase efficiency across on-premise, Cloud, and container-based environments ...

Hardware

Mellanox Technologies receives Baidu's Award for Technology Leadership, drives company to new heights in machine learning ...

Medallia deploys Mellanox Ethernet solutions to supercharge real-time analytics and achieve the efficiency of hyperscale infrastructure ...

A research project coordinated by UC3M helps reduce the cost of parallel computing ...

Supermicro shipping latest Intel Xeon Phi processor server solutions in volume with Intel Omni-Path fabric ...

Vortex laser offers hope for Moore's Law ...

Penguin Computing provides high-performance computing cluster to University of Alaska Fairbanks ...

Fifth NVM Express Plugfest participation set a record; Next event scheduled for October 2016 ...

Physicists use supercomputers to find a way of making imitation graphene from salt ...

Chip makes parallel programmes run faster with less code ...

Applications

KnuEdge and University of California San Diego to host premiere industry conference to drive next-gen machine learning performance ...

Princeton Plasma Physics Laboratory and Princeton join high-performance software project ...

Red Hat unveils JBoss Data Grid 7 as platform for real-time data analytics ...

Teradata acquires Big Data Partnership consultancy and expands open source analytics services ...

PRACE to organize Women in HPC Workshop at SC16 ...

Lawrence Livermore National Laboratory chemist named Howes Scholar ...

NSF commits $35 million to improve scientific software ...

Making Big Data DANCE(S): XSEDE project successfully tests scheduled networking of Big Data ...

New ALS gene NEK1 found by Project MinE researchers at University of Massachusetts and UMC Utrecht ...

New ALS gene C21orf2 identified by international research consortium Project MinE ...

FogHorn Systems secures $12 million in Series A funding ...

The Cloud

IBM Cloud ranked as leading Platform as a Service (PaaS) according to analyst firm ...

Oracle buys NetSuite ...

New Cloud-computing platform to further the analysis of microbial genomes ...

USFlash

Australian Government to make a $14 million investment in NCI's supercomputing and data storage capacity ...

Making Big Data DANCE(S): XSEDE project successfully tests scheduled networking of Big Data

22 Jul 2016 Pittsburgh - A project to make movement of the largest datasets more efficient has tested for the first time networking hardware components necessary for scheduling network bandwidth using Software Defined Networking (SDN). Researchers from the National Science Foundation's XSEDE collaboration of supercomputing sites reported at the XSEDE16 conference in Miami how they tested transfer of Big Data between two XSEDE sites, the Pittsburgh Supercomputing Center (PSC) and the National Institute for Computational Sciences (NICS) at the University of Tennessee.

"If you think about high-performance computing users and their end-to-end workflow, they need to do it by breaking their projects into pieces", stated Victor Hazlewood of NICS, principal author of the peer-reviewed article accompanying the presentation. "One of these pieces inevitably involves moving data into and out of where researchers do their computation."

Unfortunately for those moving the largest Big Data sets, Victor Hazlewood added, today's research wide-area networks (WAN) are more a free-for-all than their compute and storage counterparts, which are scheduled and managed. By design, the protocol software that regulates the network data flow treats all data packets as equal. When the networks are not crowded it's like an empty freeway. When they are crowded, though, the situation becomes much like a convoy of buses getting broken apart by other traffic. For large datasets traveling across the WAN, that can mean slow movement or even failures to move the data completely in a reasonable amount of time.

The NSF-funded DANCES - Developing Applications with Networking Capabilities via End-to-End SDN - is researching a kind of on-demand high-occupancy vehicle lane for large data transfers across the WAN, allowing Big Data users to specify how much bandwidth they would like to use and schedule their job so that they can be assured the data will get through in a timely fashion.

The paper presented at XSEDE16 describes the team's survey of network switching hardware that met the DANCES OpenFlow feature requirements. The paper also describes the team's successful tests using the DANCES research environment in moving large data between PSC and NICS.

"We were interested in devices that supported bandwidth reservation", stated coauthor Kathy Benninger of PSC. "Part of that has to do with adoption of the newest OpenFlow versions by switch vendors."

OpenFlow 1.3, the latest commonly deployed version of the standard network communications interface, has features that support such bandwidth management. But the problem, Kathy Benninger explained, is that "compliance" with a given version of OpenFlow doesn't require a particular network switch to support every facet of that version’s specification. The researchers had to test a number of switch products, eventually choosing the Corsa DP6410 as most fully compatible with the DANCES requirements, successfully testing the combination in a number of data-transfer scenarios between PSC and NICS. Next, the group plans to test the full set of end-to-end system functions that will be needed to support real data transfers by XSEDE users.

Victor Hazlewood expects that the SDN technology employed by DANCES may be adopted by major coordinated HPC networks such as XSEDE within the next five years, with more mainstream use in five to 10 years.

The XSEDE16 conference showcases the discoveries, innovations, challenges and achievements of those who use and support XSEDE resources and services, as well as other digital resources and services throughout the world. The theme of XSEDE16 is "Diversity, Big Data & Science at Scale: Enabling the Next-Generation of Science and Technology".

The paper is titled " Developing Applications with Networking Capabilities via End-to-End SDN (DANCES) ".
Source: Pittsburgh Supercomputing Center - PSC

Back to Table of contents

Primeur weekly 2016-08-01

Special

National Strategic Computing Initiative Executive Council to unfold Strategic Plan ...

Quantum information science is to play a key role in the U.S. National Strategic Computing Initiative ...

Focus

Lenovo is heavily investing in new HPC technology, showing its commitment to the HPC market ...

South Africa's CHPC in the picture with new Lengau supercomputer, Student Cluster Competition victory and upcoming ICRI 2016 Conference ...

Exascale supercomputing

China starts developing new-era exascale supercomputer ...

Crowd computing

IBM intensifies fight against Zika ...

Middleware

Red Hat Satellite 6.2 introduces new features to help users increase efficiency across on-premise, Cloud, and container-based environments ...

Hardware

Mellanox Technologies receives Baidu's Award for Technology Leadership, drives company to new heights in machine learning ...

Medallia deploys Mellanox Ethernet solutions to supercharge real-time analytics and achieve the efficiency of hyperscale infrastructure ...

A research project coordinated by UC3M helps reduce the cost of parallel computing ...

Supermicro shipping latest Intel Xeon Phi processor server solutions in volume with Intel Omni-Path fabric ...

Vortex laser offers hope for Moore's Law ...

Penguin Computing provides high-performance computing cluster to University of Alaska Fairbanks ...

Fifth NVM Express Plugfest participation set a record; Next event scheduled for October 2016 ...

Physicists use supercomputers to find a way of making imitation graphene from salt ...

Chip makes parallel programmes run faster with less code ...

Applications

KnuEdge and University of California San Diego to host premiere industry conference to drive next-gen machine learning performance ...

Princeton Plasma Physics Laboratory and Princeton join high-performance software project ...

Red Hat unveils JBoss Data Grid 7 as platform for real-time data analytics ...

Teradata acquires Big Data Partnership consultancy and expands open source analytics services ...

PRACE to organize Women in HPC Workshop at SC16 ...

Lawrence Livermore National Laboratory chemist named Howes Scholar ...

NSF commits $35 million to improve scientific software ...

Making Big Data DANCE(S): XSEDE project successfully tests scheduled networking of Big Data ...

New ALS gene NEK1 found by Project MinE researchers at University of Massachusetts and UMC Utrecht ...

New ALS gene C21orf2 identified by international research consortium Project MinE ...

FogHorn Systems secures $12 million in Series A funding ...

The Cloud

IBM Cloud ranked as leading Platform as a Service (PaaS) according to analyst firm ...

Oracle buys NetSuite ...

New Cloud-computing platform to further the analysis of microbial genomes ...

USFlash

Australian Government to make a $14 million investment in NCI's supercomputing and data storage capacity ...