As a consortium member, Intel plans to improve integration between iRODS, the free open source software for data virtualization, data discovery, workflow automation, and secure collaboration, and Lustre, an open source parallel distributed file system used for computing on large-scale high performance computing clusters. Membership in the consortium is a first step in offering an integrated tiered solution to Lustre end-users that allows them to easily move data sets from HPC systems into less costly long-term storage systems, where the data can be managed, shared and kept secure using iRODS.
By offering tiered storage using iRODS, administrators of HPC systems running Lustre and scientists who compute their data on these systems will be able to automate policies on when and where to move data once it is no longer needed for compute jobs, restrict and manage access to data, conduct audits, and run reports.
Scalable Lustre file systems can be part of multiple computer clusters with tens of thousands of nodes, and Lustre is capable of more than a terabyte per second of aggregate throughput. Lustre file systems are a popular choice for businesses with large data centres and data sets, including industries such as meteorology, oil and gas, life sciences, and finance.
"Having Intel and its Lustre development team as members of the iRODS Consortium gives us the opportunity to integrate the iRODS open source data management system with one of the most successful and widely used high performance distributed file system available", stated Jason Coposky, executive director of the iRODS Consortium. "We will have the opportunity to help some of the world's top scientists take control of their data and we will be able to collaborate with HPC and Lustre administrators to ensure that powerful supercomputers concentrate on computing data for science and business, rather than handling data storage and management that can be tiered onto cheaper, long-term systems."
Other members of the iRODS Consortium are Bayer, Dell/EMC, DDN, HGST, IBM, MSC, the National Institute for Computational Science at the University of Tennessee, Panasas, RENCI, Seagate, University College London, Utrecht University, and the Wellcome Trust Sanger Institute.
Consortium members, partners, iRODS users, and those interested in learning more about iRODS should plan to attend the 2017 iRODS User Group Meeting June 14 and 15 in Utrecht, the Netherlands. The meeting will feature use case presentations, live demonstrations, and open discussions about requested iRODS features. Participants representing academic, government, and commercial institutions worldwide are expected to attend. A one-day iRODS training will take place on June 13. You can learn more about the User Group Meeting at the iRODS website.