Back to Table of contents

Primeur weekly 2011-07-25

Exascale supercomputing

High-Tech Consortium launches European Technology Platform for High Performance Computing ...

The Cloud

Acer to acquire US-based iGware with US$320 million for mid- to long-term investment in Cloud technology ...

Citrix acquires Cloud.com ...

Dell announces intent to acquire data centre networking expert Force10 Networks ...

IBM Federal Community Cloud to speed agency adoption of social collaboration technologies ...

State of Vermont transforms business operations with Cloud technology from IBM and Silanis ...

LEAD Technologies releases LEADTOOLS V17.5 ...

Capgemini and Microsoft plan to offer accelerated Cloud services in 22 countries ...

NIST proposes new privacy controls for federal information systems and organisations ...

Oracle buys Ksplice ...

Therap Services expands health services infrastructure with Oracle servers ...

Oracle announces Oracle Unified Directory 11g ...

Oracle introduces Oracle VM VirtualBox 4.1 ...

Oracle WebLogic Server selected by Choice Hotels to power its Cloud infrastructure and choiceADVANTAGE SaaS application ...

Desktop Grids

8th IDGF Workshop to issue Call for Participation ...

Einstein@OSG to use resources from Open Science Grid and D-Grid ...

EuroFlash

transtec and Bright Computing enter into partnership for European HPC market ...

Bull technology supporting the University of Perpignan Via Domitia ...

Cosmological evolution of dark matter is similar to that of visible matter ...

Budding engineers build a supercomputer with the Smallpeice Trust ...

USFlash

Emerson Network Power research highlights the progress of data centre infrastructure management initiatives in the U.S. ...

New line of embedded computers from Emerson Network Power offer fast, cost-effective and lower risk solutions ...

Gigamon Solutions monitors National Center for Supercomputing Applications ...

HP helps Desjardins General Insurance Group reduce costs and risk by migrating off mainframe technology ...

HP helps enterprises advance development for the Cloud ...

HP helps enterprises scale up ...

Researchers demonstrate breakthrough storage performance for big data applications ...

Next generation IBM XIV Storage System improves efficiency for demanding application workloads ...

New IBM zEnterprise mainframe server advances smarter computing for companies and governments in emerging markets and mid-size clients around the world ...

Intel to acquire Fulcrum Microsystems ...

Jaryba and Vinetech enter into a reseller agreement to sell Jaryba SmartSuspend throughout Korea ...

Polk transforms Analytic Solutions as a Service with Oracle Exadata Database Machine and Oracle Business Intelligence Foundation ...

OSC lifts OSU land speed racer toward 400-mph goal ...

SAS' 'big analytics' tackles big data, speeds problem solving ...

CSC selects SGI to advance climate forecast modelling at NOAA ...

VMware partners embrace VMware vSphere 5 and comprehensive Cloud infrastructure suite ...

Researchers demonstrate breakthrough storage performance for big data applications

22 Jul 2011 San Jose - Researchers from IBM have demonstrated the future of large-scale storage systems by successfully scanning 10 billion files on a single system in just 43 minutes, shattering the previous record of one billion files in three hours by a factor of 37.

Growing at unprecedented scales, this advance unifies data environments on a single platform, instead of being distributed across several systems that must be separately managed. It also dramatically reduces and simplifies data management tasks, allowing more information to be stored in the same technology, rather than continuing to buy more and more storage.

In 1998, IBM Researchers unveiled a highly scalable, clustered parallel file system called General Parallel File System (GPFS), which was furthered tuned to make this breakthrough possible. GPFS represents a major advance of scaling for storage performance and capacity, while keeping management costs flat. This innovation could help organisations cope with the exploding growth of data, transactions and digitally-aware sensors and other devices that comprise Smarter Planet systems. It is ideally suited for applications requiring high-speed access to large volumes of data such as data mining to determine customer buying behaviors across massive data sets, seismic data processing, risk management and financial analysis, weather modelling and scientific research.

Today's breakthrough was achieved using GPFS running on a cluster of 10 eight core systems and solid state storage, taking 43 minutes to perform this selection. The GPFS management rules engine provides the comprehensive capabilities to service any data management task.

GPFS's advanced algorithm makes possible the full use of all processor cores on all of these machines in all phases of the task (data read, sorting and rules evaluation). GPFS exploits the solid state storage appliances with only 6.8 terabytes of capacity for excellent random performance and high data transfer rates for containing the metadata storage. The appliances sustainably perform hundreds of millions of data input-output operations, while GPFS continuously identifies, selects and sorts the right set of files among the 10 billion on the system.

"Today's demonstration of GPFS scalability will pave the way for new products that address the challenges of a rapidly growing, multi-zettabyte world", stated Doug Balog, vice president, storage platforms, IBM. "This has the potential to enable much larger data environments to be unified on a single platform and dramatically reduce and simplify data management tasks such as data placement, aging, back-up and migration of individual files."

The previous record was also set by IBM researchers at the Supercomputing 2007 conference in Reno, Nevada, where they demonstrated the ability to scan one billion files in three hours.

"Businesses in every industry are looking to the future of storage and data management as we face a problem springing from the very core of our success - managing the massive amounts of data we create on a daily basis", stated Bruce Hillsberg, director of storage systems, IBM Research - Almaden. "From banking systems to MRIs and traffic sensors, our day-to-day lives are engulfed in data. But, it can only be useful if it is effectively stored, analyzed and applied, and businesses and governments have relied on smarter technology systems as the means to manage and leverage the constant influx of data and turn it into valuable insights."

IBM Research continues to develop innovative storage technologies to help clients not only manage data proliferation, but harness data to create new services. In the past year alone, IBM storage products included over five significant storage innovations invented by IBM Research including IBM Easy Tier, Storwize V7000, Scale-out Network Attached Storage (SONAS), IBM Information Archive and IBM Long Term File System (LTFS).

As the size of digital data increased 47 percent over last year, businesses are under tremendous pressure to quickly turn data into actionable insights, but grapple with how to manage and store it all. As new applications emerge in industries from financial services to healthcare, traditional data management systems will be unable to perform common but critical storage management tasks, leaving organisations exposed to critical data loss.

Anticipating these storage challenges decades ago, researchers from IBM Research - Almaden created GPFS to help businesses cope with the exploding growth of data, transactions and digitally-aware devices on a single system. Already deployed to perform tasks like back-up, information lifecycle management, disaster recovery and content distribution, this technology's unique approach overcomes the challenge of managing unprecedented large file systems with the combination of multi-system parallelization and fast access to file system metadata stored on a solid state storage appliance.
Source: IBM

Back to Table of contents

Primeur weekly 2011-07-25

Exascale supercomputing

High-Tech Consortium launches European Technology Platform for High Performance Computing ...

The Cloud

Acer to acquire US-based iGware with US$320 million for mid- to long-term investment in Cloud technology ...

Citrix acquires Cloud.com ...

Dell announces intent to acquire data centre networking expert Force10 Networks ...

IBM Federal Community Cloud to speed agency adoption of social collaboration technologies ...

State of Vermont transforms business operations with Cloud technology from IBM and Silanis ...

LEAD Technologies releases LEADTOOLS V17.5 ...

Capgemini and Microsoft plan to offer accelerated Cloud services in 22 countries ...

NIST proposes new privacy controls for federal information systems and organisations ...

Oracle buys Ksplice ...

Therap Services expands health services infrastructure with Oracle servers ...

Oracle announces Oracle Unified Directory 11g ...

Oracle introduces Oracle VM VirtualBox 4.1 ...

Oracle WebLogic Server selected by Choice Hotels to power its Cloud infrastructure and choiceADVANTAGE SaaS application ...

Desktop Grids

8th IDGF Workshop to issue Call for Participation ...

Einstein@OSG to use resources from Open Science Grid and D-Grid ...

EuroFlash

transtec and Bright Computing enter into partnership for European HPC market ...

Bull technology supporting the University of Perpignan Via Domitia ...

Cosmological evolution of dark matter is similar to that of visible matter ...

Budding engineers build a supercomputer with the Smallpeice Trust ...

USFlash

Emerson Network Power research highlights the progress of data centre infrastructure management initiatives in the U.S. ...

New line of embedded computers from Emerson Network Power offer fast, cost-effective and lower risk solutions ...

Gigamon Solutions monitors National Center for Supercomputing Applications ...

HP helps Desjardins General Insurance Group reduce costs and risk by migrating off mainframe technology ...

HP helps enterprises advance development for the Cloud ...

HP helps enterprises scale up ...

Researchers demonstrate breakthrough storage performance for big data applications ...

Next generation IBM XIV Storage System improves efficiency for demanding application workloads ...

New IBM zEnterprise mainframe server advances smarter computing for companies and governments in emerging markets and mid-size clients around the world ...

Intel to acquire Fulcrum Microsystems ...

Jaryba and Vinetech enter into a reseller agreement to sell Jaryba SmartSuspend throughout Korea ...

Polk transforms Analytic Solutions as a Service with Oracle Exadata Database Machine and Oracle Business Intelligence Foundation ...

OSC lifts OSU land speed racer toward 400-mph goal ...

SAS' 'big analytics' tackles big data, speeds problem solving ...

CSC selects SGI to advance climate forecast modelling at NOAA ...

VMware partners embrace VMware vSphere 5 and comprehensive Cloud infrastructure suite ...