Back to Table of contents

Primeur weekly 2017-12-11

Quantum computing

NUS scientist develops 'toolboxes' for quantum cybersecurity ...

Focus on Europe

EUDAT CDI Council appoints Per Öster as Project Director of the EOSC-hub project ...

Verne Global delivers hpcDIRECT, a purpose-built, industrial scale HPC-as-a-Service platform ...

European researchers publish Quantum Software Manifesto ...

Commit2Data grant for better, faster and real-time adaptable Big Data analysis ...

Hartree Centre contributes essential computing resource to new UK National Biofilms Innovation Centre ...

Research for more efficient self-learning systems ...

Board members of the Finnish Union of University Professors visited the Netherlands eScience Center ...

Towards data storage at the single molecule level ...

Middleware

System Fabric Works and ThinkParQ become partners for highest speed Parallel File System ...

Hardware

IBM unveils industry's most advanced server designed for artificial intelligence ...

DDN's 5th annual High Performance Computing Trends survey cites complex I/O workloads as nr. 1 challenge ...

NVIDIA TITAN V transforms the PC into AI supercomputer ...

NVIDIA GPU Cloud now available to hundreds of thousands of AI researchers using NVIDIA desktop GPUs ...

New core facility boosts the University of Oregon's research computing power ...

New University of Queensland supercomputer to aid big research discoveries ...

Mellanox and NEC partner to deliver innovative high-performance and artificial intelligence platforms ...

Supermicro expands enterprise solutions portfolio with new scale-up SuperServer certified for SAP HANA ...

Applications

Cray and NERSC partnership to drive advanced AI development at scale ...

Computer simulations reveal roots of drug resistance ...

CMU receives $7.5 million in federal BRAIN initiative funding ...

New algorithm repairs corrupted digital images in one step ...

Neurons have the right shape for deep learning ...

The Cloud

Oracle open sources Kubernetes tools for serverless deployment and intelligent multi-Cloud management ...

Microsoft Azure becomes first global Cloud provider to deploy AMD EPYC ...

Neurons have the right shape for deep learning


This is an illustration of a multi-compartment neural network model for deep learning. Left: Reconstruction of pyramidal neurons from mouse primary visual cortex. Right: Illustration of simplified pyramidal neuron models. Credit: CIFAR.
4 Dec 2017 Toronto - Deep learning has brought about machines that can 'see' the world more like humans can, and recognize language. And while deep learning was inspired by the human brain, the question remains: Does the brain actually learn this way? The answer has the potential to create more powerful artificial intelligence and unlock the mysteries of human intelligence.

In a study published December 5th ineLife, CIFAR Fellow Blake Richards and his colleagues unveiled an algorithm that simulates how deep learning could work in our brains. The network shows that certain mammalian neurons have the shape and electrical properties that are well-suited for deep learning. Furthermore, it represents a more biologically realistic way of how real brains could do deep learning.

Research was conducted by Blake Richards and his graduate student Jordan Guerguiev, at the University of Toronto, Scarborough, in collaboration with Timothy Lillicrap at Google DeepMind. Their algorithm was based on neurons in the neocortex, which is responsible for higher order thought.

"Most of these neurons are shaped like trees, with 'roots' deep in the brain and 'branches' close to the surface", stated Blake Richards. "What's interesting is that these roots receive a different set of inputs than the branches that are way up at the top of the tree."

Using this knowledge of the neurons' structure, Blake Richards and Jordan Guerguiev built a model that similarly received signals in segregated compartments. These sections allowed simulated neurons in different layers to collaborate, achieving deep learning.

"It's just a set of simulations so it can't tell us exactly what our brains are doing, but it does suggest enough to warrant further experimental examination if our own brains may use the same sort of algorithms that they use in AI", Blake Richards stated.

This research idea goes back to AI pioneers Geoffrey Hinton, a CIFAR Distinguished Fellow and founder of the Learning in Machines & Brains programme, and programme Co-Director Yoshua Bengio, and was one of the main motivations for founding the programme in the first place. These researchers sought not only to develop artificial intelligence, but also to understand how the human brain learns, said Blake Richards.

In the early 2000s, Blake Richards and Timothy Lillicrap took a course with Geoffrey Hinton at the University of Toronto and were convinced deep learning models were capturing "something real" about how human brains work. At the time, there were several challenges to testing that idea. Firstly, it wasn't clear that deep learning could achieve human-level skill. Secondly, the algorithms violated biological facts proven by neuroscientists.

Now, Blake Richards and a number of researchers are looking to bridge the gap between neuroscience and AI. This paper builds on research from Yoshua Bengio's lab on a more biologically plausible way to train neural nets and an algorithm developed by Lillicrap that further relaxes some of the rules for training neural nets. The paper also incorporates research from Matthew Larkam on the structure of neurons in the neocortex. By combining neurological insights with existing algorithms, Blake Richards' team was able to create a better and more realistic algorithm simulating learning in the brain.

The tree-like neocortex neurons are only one of many types of cells in the brain. Blake Richards said future research should model different brain cells and examine how they could interact together to achieve deep learning. In the long-term, he hopes researchers can overcome major challenges, such as how to learn through experience without receiving feedback.

"What we might see in the next decade or so is a real virtuous cycle of research between neuroscience and AI, where neuroscience discoveries help us to develop new AI and AI can help us interpret and understand our experimental data in neuroscience", Blake Richards stated.

" Towards deep learning with segregated dendrites " was published ineLifeon December 5, 2017.

Source: Canadian Institute for Advanced Research

Back to Table of contents

Primeur weekly 2017-12-11

Quantum computing

NUS scientist develops 'toolboxes' for quantum cybersecurity ...

Focus on Europe

EUDAT CDI Council appoints Per Öster as Project Director of the EOSC-hub project ...

Verne Global delivers hpcDIRECT, a purpose-built, industrial scale HPC-as-a-Service platform ...

European researchers publish Quantum Software Manifesto ...

Commit2Data grant for better, faster and real-time adaptable Big Data analysis ...

Hartree Centre contributes essential computing resource to new UK National Biofilms Innovation Centre ...

Research for more efficient self-learning systems ...

Board members of the Finnish Union of University Professors visited the Netherlands eScience Center ...

Towards data storage at the single molecule level ...

Middleware

System Fabric Works and ThinkParQ become partners for highest speed Parallel File System ...

Hardware

IBM unveils industry's most advanced server designed for artificial intelligence ...

DDN's 5th annual High Performance Computing Trends survey cites complex I/O workloads as nr. 1 challenge ...

NVIDIA TITAN V transforms the PC into AI supercomputer ...

NVIDIA GPU Cloud now available to hundreds of thousands of AI researchers using NVIDIA desktop GPUs ...

New core facility boosts the University of Oregon's research computing power ...

New University of Queensland supercomputer to aid big research discoveries ...

Mellanox and NEC partner to deliver innovative high-performance and artificial intelligence platforms ...

Supermicro expands enterprise solutions portfolio with new scale-up SuperServer certified for SAP HANA ...

Applications

Cray and NERSC partnership to drive advanced AI development at scale ...

Computer simulations reveal roots of drug resistance ...

CMU receives $7.5 million in federal BRAIN initiative funding ...

New algorithm repairs corrupted digital images in one step ...

Neurons have the right shape for deep learning ...

The Cloud

Oracle open sources Kubernetes tools for serverless deployment and intelligent multi-Cloud management ...

Microsoft Azure becomes first global Cloud provider to deploy AMD EPYC ...