Back to Table of contents

Primeur weekly 2020-09-14

UCF Consortium receives open source contribution from Arm to speed access to persistent memory storage

9 Sep 2020 Santa Clara - The Unified Communication Framework (UCF), a collaboration of industry, laboratories, and academia to create production-grade communication frameworks and open standards for data-centric and high-performance applications, has announced an open source contribution of an OpenSHMEM-based I/O research extension to access persistent memory storage, in collaboration with Arm. Provided to the UCF's Open Smart Network Application Programming Interface (OpenSNAPI) project, the open source software enables Smart Networking Adapters to provide real-time access to large datasets and deliver higher application performance for latency-sensitive applications such as fraud detection, cybersecurity analysis, web-scale personalization, and Internet of Things (IoT).

New technologies continue to be developed to support the migration of the data centre architecture from the old CPU-centric concept to the data-centric concept. Utilizing the popular OpenSHMEM interface, Arm researchers have implemented an I/O interface to persistent memory (PMEM) devices to provide DRAM-like speed to storage for applications demanding high performance I/O. Utilizing UCF's OpenUCX high-performance library, which provides access to the Arm-based NVIDIA Mellanox BlueField Smart Network Adapter with attached NVDIMM-N, resulted in a direct byte-addressable persistent store for application access at near-memory speeds.

"We've seen the efficiency gains achieved by offloading network processing to smart adapters, but now we're experiencing the incredible flexibility and performance available for other offload activities, such as persistent memory storage", stated Brent Gorda, senior director of HPC, Infrastructure Line of Business, Arm. "As an active open source contributor, Arm is pleased to provide to the UCF's OpenSNAPI project this I/O extension to fuel the next wave of distributed computing applications."

OpenSNAPI is a collaboration between industry, laboratories and academia with the goal to create a standard application programming interface (API) for accessing the compute engines on the network, and specifically on the smart network adapter. OpenSNAPI allows application developers to leverage the network compute cores in parallel to the host compute cores for accelerating application runtime, and to perform operations and processing closer to the data.

"The UCF's OpenSNAPI project is helping to expand the applicability and portability of emerging use-cases for smart networking and computational storage to enhance supercomputing performance, offload security or virtualization functions, increase storage performance, and more", stated Steve Poole, UCF board and founding member. "Through open source collaboration with Arm and EMC3 at Los Alamos National Laboratory, the UCF's OpenSNAPI project is successfully showcasing the flexibility, performance and value of a new class of processing power available in the network."

The UCF consortium manages several open-source development projects, including UCX and UCX for Apache Spark.

Source: Unified Communication Framework - UCF