Back to Table of contents

Primeur weekly 2015-10-19

Special

Blue Yonder to automate enterprise decisions with machine learning ...

Videos of presentations from the Docker Workshop at ISC Cloud & Big Data conference published ...

Exascale supercomputing

Research and Markets to investigate impact of exascale computing trends in key sectors ...

Focus on Europe

Cray launches EMEA Research Lab expanding its presence in the European HPC ecosystem ...

GCS sponsors students from Technische Universität München for the Student Cluster Challenge at SC15 ...

Middleware

Moab scheduling tweak tightens Titan's workload ...

Adaptive Computing deploys Converged HPC, Cloud & Big Data for The Hospital for Sick Children & University Health Network's Princess Margaret Cancer Centre ...

Hardware

Rugged 10/40 Gigabit Ethernet switch that brings high speed networking to 3U VPX systems introduced by Curtiss-Wright ...

Asetek receives order for largest server installation to date ...

Retrofitting Rhea cluster at OLCF ...

British Antarctic Survey navigates surge of scientific research and complex climate models with DDN hybrid flash storage appliance ...

United Nations Technology Bank recognises importance of GÉANT and research and education networks ...

KBC Group delivers customers powerful banking experience driven by IBM z13 mainframe ...

Nanoelectronics researchers employ Titan for an electrifying simulation speed-up ...

Gigabyte presents its new C230 series server & workstation motherboards ...

Applications

Berkeley Lab's Yelick lauded for advances in programmability of high-performance computing systems ...

Supercoiled DNA is far more dynamic than the 'Watson-Crick' double helix ...

Ace Computers to exhibit custom HPCs and supercomputers for the Oil and Gas industry at SEG Expo ...

27 million euro for Dutch expertise centre in data-intensive scientific research ...

The Cloud

EGI opens new training infrastructure ...

VMware advances industry leading hybrid Cloud management platform and accelerates unified services delivery across public and private Clouds ...

Videos of presentations from the Docker Workshop at ISC Cloud & Big Data conference published


19 Oct 2015 Frankfurt - As part of the Primeur Live! coverage of ISC Cloud & Big Data in Frankfurt we published video coverage of all presentations of the Docker workshop. Below we provide a summary description of all presentations.

Docker introduction for HPC in 110 slides

During the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt, Holger Gantikow from Science and Computing gave an in-depth introduction into Docker. His presentation was split into 4 parts. Part I: What is Docker?; Part II: Why Docker matters; Part III: A little bit of security; Part IV: Getting started.

He explained Docker for real use, hence included also how to run applications that need an FlexLM Licence manager.

In the second part Gantikow explained why Docker matters for High-performance computing. Docker containers are more portable than traditional VMs, but have less overhead, and the performance is better. There is also a negative side to containers: the security is weaker than that of traditional VMs, although containers are catching up.

Holger Gantikow continued with examples on how to install and run Docker, and use it for applications. He concluded that Docker containers are not about virtualization 2.0. They are pretty good for portable deployment, sharing, and re-use of components. They have better performance for computing than classic virtualization, but security has to be ensured so that getting started is easy. The video recording of his presentation is about 50 minutes.

Docker & Software Stacks for building HPC clusters

During the Docker Workshop at ISC Cloud & Big Data, Frankfurt, Christian Kniep, QNIB Solutions/Gaikai presented "Docker & Software Stacks". Kniep described and demonstrated how to use QNIBTerminal, a tool to spawn HPC cluster stack using docker images. QNIBTerminal ties together service discovery, DNS interface, health check, and a key/value store. Another procect Kniep is involved in is Docker-ELK. This is a dockerfile creating ELK services (Elasticsearch/Logstash/Kibana).

HPC as a Service with OpenStack, Docker, and Trinity

Abhishek Mukherjee, ClusterVision, introduced Trinity during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt.

Trinity can function as HPC-as-a-Service, using Docker as a basis. For a SysAdmin Trinity adds the ability to partition a cluster into virtual clusters, on-demand. For a tenant of cloud resources it allows to request fully functional HPC (virtual) cluster, and to customize workspace environment. For the HPC user Trinity enables him to run HPC applications.

In the video recording Mukherjee gives a live demonstration of the deployment of Trinity.

Workflows for genome research are extremly complex - Docker and parallel processing come to the rescue

Maria Chatzou introduced Nextflow during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt. Nextflow is a system to manage scientific workflows for genome analysis. Scientific workflows are complex and very big. Chatzou citesNature: "Storing and processing genome data will exceed the computing challenges of running YouTube and Twitter, biologists warn". The workflows consist of multiple third party software components with many dependencies (libraries, scripts, tools, etc.). They are research tools that frequently change, so there is a need for flexibility. However, Chatzou notes that maintaining a scientific production chain, with controlled versions, within a standard production environment is impossible in practice. That is why NextFlow was developed.

Nextflow has several useful features, including automatic Parallelization; Resumption of Pipelines; and Polyglot. It takes care of virtualization & packaging, is portable and is platform agnostic. Nextflow also supports sharing and collaboration. In the distributed model of Nextflow, Docker is used to package tools and apps.

In the video recording of the presentation, Maria Chatzou demonstrates the workings of Nextflow live.

UberCloud provides engineering HPC-in-the-Cloud services using Docker

The UberCloud is an online marketplace with 34 stores providing computational engineering services. UberCloud's Wolfgang Gentzsch explained in the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt they used Docker containers right from the start.

The latest development are the UberCloud application containers. These are based on Docker, but enhanced for engineering & scientific applications. They contain software packages designed to deliver the tools that an engineer needs.

The UberCloud application containers are ready to execute in an instant. Software is pre-installed, configured, and tested, and running on bare metal, without loss of performance. Currently there are UberCloud application containers for many ISV application packages, including ANSYS, CD-adapco, Numeca, OpenFOAM, Gromacs, and Scilab.

Meanwhile UberCloud has built a complete Dockerized technical computing ecosystem.

In the video recording of the presentation, Wolfgang Gentzsch demonstrates live an application of OpenFoam on the UberCloud.

HPC Software Development with Docker

Johann Weging explained and demoed how to develop software with Docker during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt. Topics included configuration management, deployment and continuous integration. Weging also explained how to set up a system for HPC applications, which can be time consuming. Waiting for resources takes a long time and there are long test/modify cycles.

In most part of the 20 minute video Weging demonstrates live how to set up and deploy a software development system. Prepared to see a lot of Unix command line there.

Ad Emmen

Back to Table of contents

Primeur weekly 2015-10-19

Special

Blue Yonder to automate enterprise decisions with machine learning ...

Videos of presentations from the Docker Workshop at ISC Cloud & Big Data conference published ...

Exascale supercomputing

Research and Markets to investigate impact of exascale computing trends in key sectors ...

Focus on Europe

Cray launches EMEA Research Lab expanding its presence in the European HPC ecosystem ...

GCS sponsors students from Technische Universität München for the Student Cluster Challenge at SC15 ...

Middleware

Moab scheduling tweak tightens Titan's workload ...

Adaptive Computing deploys Converged HPC, Cloud & Big Data for The Hospital for Sick Children & University Health Network's Princess Margaret Cancer Centre ...

Hardware

Rugged 10/40 Gigabit Ethernet switch that brings high speed networking to 3U VPX systems introduced by Curtiss-Wright ...

Asetek receives order for largest server installation to date ...

Retrofitting Rhea cluster at OLCF ...

British Antarctic Survey navigates surge of scientific research and complex climate models with DDN hybrid flash storage appliance ...

United Nations Technology Bank recognises importance of GÉANT and research and education networks ...

KBC Group delivers customers powerful banking experience driven by IBM z13 mainframe ...

Nanoelectronics researchers employ Titan for an electrifying simulation speed-up ...

Gigabyte presents its new C230 series server & workstation motherboards ...

Applications

Berkeley Lab's Yelick lauded for advances in programmability of high-performance computing systems ...

Supercoiled DNA is far more dynamic than the 'Watson-Crick' double helix ...

Ace Computers to exhibit custom HPCs and supercomputers for the Oil and Gas industry at SEG Expo ...

27 million euro for Dutch expertise centre in data-intensive scientific research ...

The Cloud

EGI opens new training infrastructure ...

VMware advances industry leading hybrid Cloud management platform and accelerates unified services delivery across public and private Clouds ...