Back to Table of contents

Primeur weekly 2020-05-25

Focus

InsideHPC's Rich Brueckner passes away at 58 ...

Only half of the EuroHPC countries confirmed they will participate in co-funding EuroHPC projects ...

Quantum computing

Atos and CSC empower the Finnish quantum research community with Atos Quantum Learning Machine ...

Quantum Hall effect 'reincarnated' in 3D topological materials ...

University of California Los Angeles physicists develop world's best quantum bits ...

Quantum leap: Photon discovery is a major step toward at-scale quantum technologies ...

Focus on Europe

HPC-AI Advisory Council and ISC Group continue Annual Student Cluster Competition as an online event ...

ELASTIC software architecture advances urban mobility in Florence ...

Middleware

Waste energy of LUMI supercomputer produces 20 percent of the district heat of Kajaani ...

Italy-based Do IT Systems becomes Bright Services Partner ...

Hardware

Microsoft claims to have built new supercomputer for OpenAI ...

New York University and IBM research takes electrons for a spin in moving toward more efficient, higher density data ...

GRC wins Phase I of AFWERX Programme ...

Samsung Electronics expands its foundry capacity with a new production line in Pyeongtaek, Korea ...

HPE reports Q2 results ...

Intersect360 Research invites your input in an all new survey ...

Applications

Scientists seeking therapies for deadly bacterial disease ...

2020 OLCF User Meeting will happen remote together ...

Summit supercomputer is mining for COVID-19 connections ...

A new method for unraveling complex gene interactions ...

ACM honours computing innovators for advances in research, education and industry ...

Discovery about the edge of fusion plasma could help realize fusion power ...

Superconductors with 'zeitgeist' - When materials differentiate between past and future ...

NIST team builds hybrid quantum system by entangling molecule with atom ...

BIOS IT partners with Panasas to deliver ActiveStor Ultra turnkey HPC storage appliances ...

Supercomputer model simulations reveal cause of Neanderthal extinction ...

Pittsburgh Supercomputing Center is supporting COVID-19 research ...

The Cloud

New technique separates industrial noise from natural seismic signals ...

Lyell Immunopharma goes all-in on AWS as its Cloud provider ...

WekaIO furthers Weka AI by integrating with deep learning pipeline management solution from Valohai ...

WekaIO furthers Weka AI by integrating with deep learning pipeline management solution from Valohai


20 May 2020 Campbell - WekaIO, an Advanced Technology Partner in the Amazon Web Services (AWS) Partner Network (APN) and an innovation expert in high-performance and scalable file storage, has announced its integration with the deep learning pipeline management solution from Valohai, a Weka Innovation Network (WIN) partner. The announcement underpins Weka's commitment to empower Data Scientists and Chief Data Officers to manage and prioritize data science pipelines. The tools from Valohai are supported in an Amazon Virtual Private Cloud (Amazon VPC), and available in AWS Marketplace.

"New workloads are driving the need for modern foundational architectures, and the recently launched Weka AI offers a transformative solution framework for Accelerated DataOps", stated Shailesh Manjrekar, head of AI and strategic alliances at Weka. "Our partnership with Valohai and our integration with its Deep Learning Pipeline Management tools expand Weka AI's capabilities to offer Explainable AI (XAI). This is a critical factor for use cases with a social impact, including autonomous driving, health care, and genomics."

Integration with solutions from technology alliance partners such as Valohai enhances the power of Weka AI. Underpinned by the Weka File System (WekaFS), Weka AI now provides a production-ready solution where the entire AI data pipeline workflow, from data ingestion to batch feature extraction, training, hyperparameter optimization, and finally to inference and versioning, can run on the same storage platform, whether on-premises or on AWS. This is only possible because of the excellent mixed workload performance and the data management and governance abilities provided by WekaFS.

"Machine Learning (ML) gives businesses a competitive advantage, but while ML is hard, real-world ML is much harder", stated Eero Laaksonen, chief executive officer at Valohai. "A real-world ML system is 95% enabling code, with only 5% actual ML code that creates business value, so the question becomes how to ensure that efforts are focused on that 5%. With Valohai, businesses can focus on data science, as it handles everything else. It is as simple as pointing to your code and data and hitting 'run'. The seamless integration between our DLMS solution and Weka’s powerful snap-to-object capabilities offer a quick, zero-set-up infrastructure for DataOps. This further helps businesses to build models 10x faster, regain 35% of lost cloud costs, and free their DataOps teams with automated ML orchestration, data management, and data mobility. It is a win-win."

Weka AI, with Valohai DLMS, enables hybrid workflows, where data scientists can employ Jupyter Notebook or the Valohai GUI, either on-premise or on AWS, to do data transformation, model training, hyperparameter optimization, and inference in a Kubernetes-orchestrated environment. Valohai DLMS seamlessly integrates with WekaFS, which leverages i3.n Amazon Elastic Compute Cloud (Amazon EC2) instances with Non-Volatile Memory Express (NVMe) flash for the performance tier and extends the file namespace over Amazon Simple Storage Service (Amazon S3) buckets for the capacity tier.

Valohai DLMS also takes file namespace snapshots when running experiments and stores them for data versioning. These versions can then be rehydrated anytime to reproduce an experiment and provides the required explainability and transparency. Weka AI provides security and governance with end-to-end encryption of the pipeline and also provides integration with leading key management solutions such as, HashiCorp Vault.

Shailesh Manjrekar added: "Weka AI, when deployed with Valohai on AWS, builds upon our success for accelerating Genomics, Fintech, and AI/ML/DL data pipelines. Customers use Weka on AWS for outstanding performance as we can showcase 100GB/sec of throughput with 5 million 4KB IOPS for a 16xi3en cluster, all with less than 250-microsecond latency."

Source: WekaIO

Back to Table of contents

Primeur weekly 2020-05-25

Focus

InsideHPC's Rich Brueckner passes away at 58 ...

Only half of the EuroHPC countries confirmed they will participate in co-funding EuroHPC projects ...

Quantum computing

Atos and CSC empower the Finnish quantum research community with Atos Quantum Learning Machine ...

Quantum Hall effect 'reincarnated' in 3D topological materials ...

University of California Los Angeles physicists develop world's best quantum bits ...

Quantum leap: Photon discovery is a major step toward at-scale quantum technologies ...

Focus on Europe

HPC-AI Advisory Council and ISC Group continue Annual Student Cluster Competition as an online event ...

ELASTIC software architecture advances urban mobility in Florence ...

Middleware

Waste energy of LUMI supercomputer produces 20 percent of the district heat of Kajaani ...

Italy-based Do IT Systems becomes Bright Services Partner ...

Hardware

Microsoft claims to have built new supercomputer for OpenAI ...

New York University and IBM research takes electrons for a spin in moving toward more efficient, higher density data ...

GRC wins Phase I of AFWERX Programme ...

Samsung Electronics expands its foundry capacity with a new production line in Pyeongtaek, Korea ...

HPE reports Q2 results ...

Intersect360 Research invites your input in an all new survey ...

Applications

Scientists seeking therapies for deadly bacterial disease ...

2020 OLCF User Meeting will happen remote together ...

Summit supercomputer is mining for COVID-19 connections ...

A new method for unraveling complex gene interactions ...

ACM honours computing innovators for advances in research, education and industry ...

Discovery about the edge of fusion plasma could help realize fusion power ...

Superconductors with 'zeitgeist' - When materials differentiate between past and future ...

NIST team builds hybrid quantum system by entangling molecule with atom ...

BIOS IT partners with Panasas to deliver ActiveStor Ultra turnkey HPC storage appliances ...

Supercomputer model simulations reveal cause of Neanderthal extinction ...

Pittsburgh Supercomputing Center is supporting COVID-19 research ...

The Cloud

New technique separates industrial noise from natural seismic signals ...

Lyell Immunopharma goes all-in on AWS as its Cloud provider ...

WekaIO furthers Weka AI by integrating with deep learning pipeline management solution from Valohai ...