During the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt, Holger Gantikow from Science and Computing gave an in-depth introduction into Docker. His presentation was split into 4 parts. Part I: What is Docker?; Part II: Why Docker matters; Part III: A little bit of security; Part IV: Getting started.
He explained Docker for real use, hence included also how to run applications that need an FlexLM Licence manager.
In the second part Gantikow explained why Docker matters for High-performance computing. Docker containers are more portable than traditional VMs, but have less overhead, and the performance is better. There is also a negative side to containers: the security is weaker than that of traditional VMs, although containers are catching up.
Holger Gantikow continued with examples on how to install and run Docker, and use it for applications. He concluded that Docker containers are not about virtualization 2.0. They are pretty good for portable deployment, sharing, and re-use of components. They have better performance for computing than classic virtualization, but security has to be ensured so that getting started is easy. The video recording of his presentation is about 50 minutes.
During the Docker Workshop at ISC Cloud & Big Data, Frankfurt, Christian Kniep, QNIB Solutions/Gaikai presented "Docker & Software Stacks". Kniep described and demonstrated how to use QNIBTerminal, a tool to spawn HPC cluster stack using docker images. QNIBTerminal ties together service discovery, DNS interface, health check, and a key/value store. Another procect Kniep is involved in is Docker-ELK. This is a dockerfile creating ELK services (Elasticsearch/Logstash/Kibana).
Abhishek Mukherjee, ClusterVision, introduced Trinity during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt.
Trinity can function as HPC-as-a-Service, using Docker as a basis. For a SysAdmin Trinity adds the ability to partition a cluster into virtual clusters, on-demand. For a tenant of cloud resources it allows to request fully functional HPC (virtual) cluster, and to customize workspace environment. For the HPC user Trinity enables him to run HPC applications.
In the video recording Mukherjee gives a live demonstration of the deployment of Trinity.
Maria Chatzou introduced Nextflow during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt. Nextflow is a system to manage scientific workflows for genome analysis. Scientific workflows are complex and very big. Chatzou citesNature: "Storing and processing genome data will exceed the computing challenges of running YouTube and Twitter, biologists warn". The workflows consist of multiple third party software components with many dependencies (libraries, scripts, tools, etc.). They are research tools that frequently change, so there is a need for flexibility. However, Chatzou notes that maintaining a scientific production chain, with controlled versions, within a standard production environment is impossible in practice. That is why NextFlow was developed.
Nextflow has several useful features, including automatic Parallelization; Resumption of Pipelines; and Polyglot. It takes care of virtualization & packaging, is portable and is platform agnostic. Nextflow also supports sharing and collaboration. In the distributed model of Nextflow, Docker is used to package tools and apps.
In the video recording of the presentation, Maria Chatzou demonstrates the workings of Nextflow live.
The UberCloud is an online marketplace with 34 stores providing computational engineering services. UberCloud's Wolfgang Gentzsch explained in the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt they used Docker containers right from the start.
The latest development are the UberCloud application containers. These are based on Docker, but enhanced for engineering & scientific applications. They contain software packages designed to deliver the tools that an engineer needs.
The UberCloud application containers are ready to execute in an instant. Software is pre-installed, configured, and tested, and running on bare metal, without loss of performance. Currently there are UberCloud application containers for many ISV application packages, including ANSYS, CD-adapco, Numeca, OpenFOAM, Gromacs, and Scilab.
Meanwhile UberCloud has built a complete Dockerized technical computing ecosystem.
In the video recording of the presentation, Wolfgang Gentzsch demonstrates live an application of OpenFoam on the UberCloud.
Johann Weging explained and demoed how to develop software with Docker during the Docker workshop at the ISC Cloud & Big Data Conference in Frankfurt. Topics included configuration management, deployment and continuous integration. Weging also explained how to set up a system for HPC applications, which can be time consuming. Waiting for resources takes a long time and there are long test/modify cycles.
In most part of the 20 minute video Weging demonstrates live how to set up and deploy a software development system. Prepared to see a lot of Unix command line there.