Back to Table of contents

Primeur weekly 2017-02-20

Focus

HPC expert Genias Benelux to show its skillful expertise in brandnew website ...

Are billion Euro Flagships the right way to finance innovative areas like graphene, human brain research and quantum computing? ...

Exascale supercomputing

Advanced fusion code led by PPPL selected to participate in Early Science Programmes on three new DOE Office of Science pre-exascale supercomputers ...

Focus on Europe

From robotics to particle physics: Data analytics gets the spotlight in Distinguished Talk series at ISC 2017 ...

A new spin on electronics ...

Data mining tools for personalized cancer treatment ...

Why host HPC in Iceland to tackle Big Data for life sciences at Earlham Insititute ...

Biological experiments become transparent - anywhere, any time ...

Middleware

IBM delivers new platform to help clients address storage challenges at massive scale ...

Hewlett Packard Enterprise unveils most significant 3PAR Flash storage innovations to date ...

Hardware

Tokyo Institute of Technology partners with DDN on Tsubame3.0 to build forward-looking AI and Big Data computing infrastructure ...

Mellanox demonstrates four times improvement in crypto performance with Innova IPsec 40G Ethernet network adapter ...

Supermicro launches BigTwin - the industry's highest performing Twin multi-node system supporting the full range of CPUs, maximum memory and all-flash NVMe ...

Applications

Researchers catch extreme waves with higher-resolution modelling ...

Researchers are creating software to 'clean' large datasets, making it easier for scientists and the public to use Big Data ...

Designing new materials from 'small' data ...

Success by deception ...

DNA computer brings 'intelligent drugs' a step closer ...

'Lossless' metamaterial could boost efficiency of lasers and other light-based devices ...

Perimeter Institute researchers apply machine learning to condensed matter physics ...

When treating brain aneurysms, two isn't always better than one ...

Real-time MRI analysis powered by supercomputers ...

Analyzing data for transportation systems using TACC's Rustler, XSEDE ECSS support ...

NCSA facilitates performance comparisons with China's nr. 1 supercomputer ...

IBM delivers Watson for cyber security to power cognitive security operations centres ...

The Cloud

Optimizing data centre placement and network design to strengthen Cloud computing ...

Dutch start-up solution impacts data centres ...

OpenFog Consortium releases landmark reference architecture for Fog computing ...

IBM brings machine learning to the private Cloud ...

IBM accelerates hybrid Cloud adoption by enabling channel partners to offer VMware solutions ...

Oracle launches Cloud service to help organisations integrate disparate data and drive real-time analytics ...

Analyzing data for transportation systems using TACC's Rustler, XSEDE ECSS support

Preliminary visualization of trip-level data after processing on Rustler.13 Feb 2017 Austin - In the next 10 years you are going to see some form of autonomous or connected vehicles on the streets. Natalia Ruiz Juri, a research associate with the University of Texas at Austin's Center for Transportation Research (CTR) is fairly certain of this. She is one of many researchers at CTR and The University of Texas at Austin (UT Austin) who are studying the wide range of technical, social and policy aspects of connected and autonomous vehicle (CAV) technologies.

Fully autonomous vehicles or driverless cars are capable of sensing their environment and navigating without human input. They can detect surroundings using a variety of techniques such as radar, lidar, GPS, odometry, and computer vision. Similarly, connected vehicles (CVs) are vehicles that can exchange messages containing location and other safety-related information with other vehicles, and with devices affixed to roadside infrastructure.

CVs share information in the form of Basic Safety Messages (BSMs) with other vehicles and the infrastructure; these include vehicle position, speed and breaking status. Such real-time feedback and information exchange between vehicles is expected to greatly enhance safety, and it opens the door to several possibilities in traffic management.

For example, vehicles could talk to other vehicles that are much further ahead and get warned about congestion or dangerous conditions, thereby allowing a driver to make strategic decisions and take a different path.

Additionally, vehicles could also talk to infrastructure, such as an intersection light, which might be capable of tracking the number of vehicles passing through and potentially adjusting the signal timing plan accordingly. The advent of CVs would therefore have huge promise in improving traffic management and the overall utilization of transportation infrastructure, particularly if vehicle connectivity is considered along with automation.

While the basic goal of CVs, in particular, is safety - experts hypothesize up to 80 percent less accidents in the future - the data generated by CVs has an enormous potential to support transportation planning and operations.

At this point researchers are still exploring diverse datasets. A number of connected vehicle test beds and autonomous vehicles test sites have been planned, or are already in place. Texas is part of one of the 10 US-Department of Transportation-designated autonomous vehicle proving grounds, and research sponsored by other agencies, such as TxDOT and the North Central Texas Council of Governments is also happening at UT Austin.

"The volume and complexity of CV data are tremendous and present a Big Data challenge for the transportation research community", Natalia Ruiz Juri stated. While there is uncertainty in the characteristics of the data that will eventually be available, the ability to efficiently explore existing datasets is paramount.

Natalia Ruiz Juri and her colleagues, including Chandra Bhat, James Kuhr and Jackson Archer, were interested in exploring the most comprehensive data set released to date - the Safety Pilot Model Deployment (SPMD) data, produced by a study conducted by the University of Michigan Transportation Research Institute and the National Highway Traffic Safety Administration.

However, to get started they needed help using computational resources. They turned to the Texas Advanced Computing Center (TACC), also at UT Austin, and a key partner in the Extreme Science and Engineering Discovery Environment (XSEDE), the most advanced, powerful, and robust collection of integrated advanced digital resources and services in the world. Through XSEDE, Natalia Ruiz Juri and team took advantage of the Extended Collaborative Support Services (ECSS) programme, and the TACC experts within the programme, to make these resources easier to use and to help more people use them.

TACC ECSS experts Weijia Xu and Amit Gupta were able to help Natalia Ruiz Juri and her colleagues figure out how to use very large datasets on supercomputers like Rustler, TACC's experimental system for exploring new storage and data compute techniques and technologies.

Natalia Ruiz Juri and her colleagues compared efforts to build scalable solutions for CV data analysis using Hive, an open-source data warehouse application that supports distributed queries. The data included approximately 2,700 cars, trucks and transit buses whose activities were logged through on-board sensors over a two month period.

"Hive is an ideal choice in this particular use case since it not only offers scalability and performance but also has a SQL-like interface", Weijia Xu stated, referring to Structured Query Language used to manage data. "It is similar to PostgreSQL which the research team is already familiar with."

According to Natalia Ruiz Juri, using Rustler is a huge time-saver because it lets them play with the data and see what it looks like without spending hours waiting for a query to complete.

As a researcher, Natalia Ruiz Juri said one of the challenges she faces is not knowing which system to use on a particular model for a particular dataset. This is one of the many ways that Weijia Xu and Amit Gupta were able to help. They developed an automated methodology to understand how each system is expected to perform based on the characteristics of the network. For this work they used Rustler, but soon they plan to move the data to Wrangler, an XSEDE-allocated resource.

"Natalia and her colleagues were trying to make sense of the data", Amit Gupta stated. "It's unfiltered data from real people capturing their movement patterns across the city. All of this data was sampled at 10 times per second - speed data, when a person used their brakes, when they used their windshield wipers etc. - so it's a lot of information and nobody has completely figured out what to do with it. Natalia and her team are trying to validate, and in some cases possibly break through, some of the assumptions that they traditionally made in their field with respect to traffic patterns."

In addition to determining which system to use for which model, Weijia Xu and Amit Gupta also helped Natalia Ruiz Juri and colleagues create a friendly user interface to remove some of the hurdles of using a command line. If you don't have an interface, the researcher has to come up with something manually and they may not have time or funding to do that, especially when in exploration mode. "The interface gave us the opportunity to look at this data now instead of, say, two years down the line in the project", Natalia Ruiz-Juri stated.

"The XSEDE ECSS programme has been great for us", Natalia Ruiz Juri stated. "We get together and we talk about projects and research in general. Amit and Weijia have started to understand more about what we are doing, so for me the best part is not when we know what we want and they help us, but when they understand enough of what we're doing and can come up with new ideas on their own. We've been working together for over three years now on different projects."

The goal is to enable their research exploration by leveraging HPC tools and infrastructure, according to Amit Gupta. Due to the scale of such resources available at TACC, they are able to iterate through their analysis cycle much quicker and converge towards conclusions faster. It also enables them to attempt new simulation experiments that would overload their computational resources or take prohibitively long to run.

"I enjoy working on this project very much", Amit Gupta stated. "It's one of my favorite projects. It's a very challenging and interesting application of computer science to a real world problem."

One of the challenges with research in this field is that connected and autonomous vehicles can be disruptive, according to Natalia Ruiz Juri. How do we anticipate what's going to happen in the future when this type of technology can change not only transportation system performance, but also travel choices and behaviour?

How are people going to react to this technology? Are they going to purchase more cars, fewer cars? Are they going to travel further? Are they not going to care about travel time any longer so they move further away from downtown?

Researchers want to understand how they need to modify existing models so that they can consider all these complex, interrelated impacts, when assessing the effects of CAV technologies into the future. Advanced models require significant computational resources, and TACC experts have already supported CTR in the use of HPC for their simulations.

"It would have been very hard without the help of Amit or Weijia to be able to have visibility and access to HPC from the interface of a preexisting code that we use for modeling, and which may be central to future research in CVs. They helped us a lot in terms of how to access the systems, how to set up log-in, writing scripts, authentication, creating accounts, and much more", Natalia Ruiz Juri stated.

Using models to test all of the hypotheses and questions can transform the way we think about living and travelling.

"I think that vehicle connectivity is going to happen relatively soon and it's going to make travel safer - it's something to look forward to", Natalia Ruiz Juri stated. "It also gives us the opportunity to collect a lot of data so we can look at operating transportation systems differently. It has huge potential for safety and traffic operations. Automated vehicles are an exciting possibility that can truly transform how we travel, and lead to major changes in lifestyle choices and decisions."
Source: University of Texas at Austin, Texas Advanced Computing Center - TACC

Back to Table of contents

Primeur weekly 2017-02-20

Focus

HPC expert Genias Benelux to show its skillful expertise in brandnew website ...

Are billion Euro Flagships the right way to finance innovative areas like graphene, human brain research and quantum computing? ...

Exascale supercomputing

Advanced fusion code led by PPPL selected to participate in Early Science Programmes on three new DOE Office of Science pre-exascale supercomputers ...

Focus on Europe

From robotics to particle physics: Data analytics gets the spotlight in Distinguished Talk series at ISC 2017 ...

A new spin on electronics ...

Data mining tools for personalized cancer treatment ...

Why host HPC in Iceland to tackle Big Data for life sciences at Earlham Insititute ...

Biological experiments become transparent - anywhere, any time ...

Middleware

IBM delivers new platform to help clients address storage challenges at massive scale ...

Hewlett Packard Enterprise unveils most significant 3PAR Flash storage innovations to date ...

Hardware

Tokyo Institute of Technology partners with DDN on Tsubame3.0 to build forward-looking AI and Big Data computing infrastructure ...

Mellanox demonstrates four times improvement in crypto performance with Innova IPsec 40G Ethernet network adapter ...

Supermicro launches BigTwin - the industry's highest performing Twin multi-node system supporting the full range of CPUs, maximum memory and all-flash NVMe ...

Applications

Researchers catch extreme waves with higher-resolution modelling ...

Researchers are creating software to 'clean' large datasets, making it easier for scientists and the public to use Big Data ...

Designing new materials from 'small' data ...

Success by deception ...

DNA computer brings 'intelligent drugs' a step closer ...

'Lossless' metamaterial could boost efficiency of lasers and other light-based devices ...

Perimeter Institute researchers apply machine learning to condensed matter physics ...

When treating brain aneurysms, two isn't always better than one ...

Real-time MRI analysis powered by supercomputers ...

Analyzing data for transportation systems using TACC's Rustler, XSEDE ECSS support ...

NCSA facilitates performance comparisons with China's nr. 1 supercomputer ...

IBM delivers Watson for cyber security to power cognitive security operations centres ...

The Cloud

Optimizing data centre placement and network design to strengthen Cloud computing ...

Dutch start-up solution impacts data centres ...

OpenFog Consortium releases landmark reference architecture for Fog computing ...

IBM brings machine learning to the private Cloud ...

IBM accelerates hybrid Cloud adoption by enabling channel partners to offer VMware solutions ...

Oracle launches Cloud service to help organisations integrate disparate data and drive real-time analytics ...