Back to Table of contents

Primeur weekly 2017-09-04

Special

ETP4HPC to lay out roadmap for tangible HPC products within Europe ...

Focus

Primeur Magazine team unveils the largest directory of HPC organisations in the universe ...

Primeur Magazine team unveils the largest directory of HPC organisations in the universe ...

Quantum computing

First quantum annealing computer in the U.S. to have more than 2000 qubits installed and operational ...

Is the world on the brink of a computing revolution? - Quantum computing at the 5th Heidelberg Laureate Forum ...

Focus on Europe

Smart tomographic sensors control industrial processes of tomorrow ...

DeepL to launch DeepL Translator ...

Middleware

Learning Database speeds queries from hours to seconds ...

Unifying statistics, computer science, and applied mathematics ...

New theorems help robots to correct errors on-the-fly and learn from each other ...

Ace Computers delivers BeeGFS parallel file system for supercomputers ...

Embry-Riddle Aeronautical University acquires Cray CS cluster supercomputer to advance aerospace research ...

Hardware

ScaleMP completes $10 million funding round to accelerate growth ...

Optalysys and Earlham Institute demonstrate results of breakthrough optical processing for sequence alignment ...

University of Southern Mississippi gains campus-wide HPC cluster for research ...

Applications

New boarding procedures, smaller cabin size may limit infection on planes ...

Shaden Smith and Yang You announced as recipients of 2017 ACM/IEEE-CS George Michael Memorial HPC Fellowships ...

A new cosmic lab to view the Big Bang movie ...

Optical control of magnetic memory - New insights into fundamental mechanisms ...

Machine-learning earthquake prediction in lab shows promise ...

Caching system could make data centres more energy efficient ...

Supercomputing the weather with Thor ...

The Cloud

VMware and Dell EMC partner to deliver first data protection solution for VMware Cloud on AWS ...

Oracle expands IoT Cloud portfolio, enabling customers to accelerate intelligence and ROI from connected assets ...

VMware advances software to help customers modernize data centres ...

Fujitsu and VMware extend global partnership to empower organisations' digital transformations ...

VMware and AWS announce initial availability of VMware Cloud on AWS ...

VMware and Pivotal launch Pivotal Container Service (PKS) and collaborate with Google Cloud to bring Kubernetes to enterprise customers ...

Learning Database speeds queries from hours to seconds

31 Aug 2017 Ann Arbor - A tool that makes large databases work smarter, not harder, could unlock the potential of Big Data to drive medical research, inform business decisions and speed up a slew of other applications that today are mired in a worldwide data glut.

University of Michigan researchers developed software called Verdict that enables existing databases to learn from each query a user submits, finding accurate answers without trawling through the same data again and again. Verdict allows databases to deliver answers more than 200 times faster while maintaining 99 percent accuracy. In a research environment, that could mean getting answers in seconds instead of hours or days.

When speed isn't required, it can be set to save electricity, using 200 times less than a traditional database. This could lead to substantial power savings, the researchers say, as data centres gobble up a growing share of the world's electricity.

Verdict is believed to be the first working example in a new field of research called "database learning".

"Databases have been following the same paradigm for the past 40 years", stated Barzan Mozafari, the Morris Wellman Faculty Development Assistant Professor of Computer Science and Engineering. "You submit a query, it does some work and provides an answer. When a new query comes in, it starts over. All the work from previous queries is wasted."

Verdict changes that. It relies on advanced statistical principles, using past question-and-answer pairs to infer where the answers to future queries are likely to lie.

The researchers say the innovation can't come soon enough, as the digital world is up to well over 1 billion gigabytes of stored data - everything from genomic data to hospital records and online shopping histories. And new data is streaming in far more quickly than systems can process it. Increased processing power won't solve the problem, as the rate of new data generation is increasing faster than processing power.

Meanwhile, data has become a driver for life-saving medical research and sophisticated business decision-making. It's increasingly being tasked with not just finding answers, but also uncovering new ideas that can drive the direction of research. Medical researchers are turning databases loose on massive stockpiles of patient data to find buried connections between health status and disease. Retailers like Amazon are taking a similar approach to find precisely what motivates customers to buy and how to optimize supply chains, while online ad firms use data-driven algorithms to serve up the right ad at the right moment.

Such research can involve hundreds or thousands of simultaneous queries, and waiting hours for an answer is more than just an inconvenience. Studies have shown that even a short delay can hamper productivity and stifle innovation.

Verdict is what's known as a "thin layer" - a small, nimble piece of software that can be placed in front of any existing database. At first, it simply stores queries that go in and out of the database, compiling them into what's called a query synopsis.

After storing a given number of queries, it goes into action, breaking each query up into component parts called snippets and using them to build a mathematical model of questions and answers. When a new query comes in, it uses that model to point the database to a certain subset of data where the answer is likely to be found. In some cases, it can even find an answer using only the model - without looking at the database at all.

Verdict itself uses minimal computing resources, and Barzan Mozafari, along with research fellow Youngjoo Park, has demonstrated that it doesn't slow performance. It also enables users to tailor the balance between speed and accuracy to fit individual applications. Barzan Mozafari believes a commercial product is likely a few years off.

"We've really just scratched the surface of what database learning can do", he stated. "The important thing is that we've turned the mechanics of the database upside down. Instead of just additional work, each query is now an opportunity to learn and make the database work better."

The project is detailed in a study titled " ".

Source: University of Michigan

Back to Table of contents

Primeur weekly 2017-09-04

Special

ETP4HPC to lay out roadmap for tangible HPC products within Europe ...

Focus

Primeur Magazine team unveils the largest directory of HPC organisations in the universe ...

Primeur Magazine team unveils the largest directory of HPC organisations in the universe ...

Quantum computing

First quantum annealing computer in the U.S. to have more than 2000 qubits installed and operational ...

Is the world on the brink of a computing revolution? - Quantum computing at the 5th Heidelberg Laureate Forum ...

Focus on Europe

Smart tomographic sensors control industrial processes of tomorrow ...

DeepL to launch DeepL Translator ...

Middleware

Learning Database speeds queries from hours to seconds ...

Unifying statistics, computer science, and applied mathematics ...

New theorems help robots to correct errors on-the-fly and learn from each other ...

Ace Computers delivers BeeGFS parallel file system for supercomputers ...

Embry-Riddle Aeronautical University acquires Cray CS cluster supercomputer to advance aerospace research ...

Hardware

ScaleMP completes $10 million funding round to accelerate growth ...

Optalysys and Earlham Institute demonstrate results of breakthrough optical processing for sequence alignment ...

University of Southern Mississippi gains campus-wide HPC cluster for research ...

Applications

New boarding procedures, smaller cabin size may limit infection on planes ...

Shaden Smith and Yang You announced as recipients of 2017 ACM/IEEE-CS George Michael Memorial HPC Fellowships ...

A new cosmic lab to view the Big Bang movie ...

Optical control of magnetic memory - New insights into fundamental mechanisms ...

Machine-learning earthquake prediction in lab shows promise ...

Caching system could make data centres more energy efficient ...

Supercomputing the weather with Thor ...

The Cloud

VMware and Dell EMC partner to deliver first data protection solution for VMware Cloud on AWS ...

Oracle expands IoT Cloud portfolio, enabling customers to accelerate intelligence and ROI from connected assets ...

VMware advances software to help customers modernize data centres ...

Fujitsu and VMware extend global partnership to empower organisations' digital transformations ...

VMware and AWS announce initial availability of VMware Cloud on AWS ...

VMware and Pivotal launch Pivotal Container Service (PKS) and collaborate with Google Cloud to bring Kubernetes to enterprise customers ...