Deep Bayesian Reinforcement Learning -- Unifying Perception, Planning, and Control

Introduction

For robots to assist humanity in homes or hospitals, the capability to manipulate diverse objects is imperative. So far, however, robotic manipulation technology has struggled in managing the uncertainty and unstructuredness that characterize human environments.

Machine Learning Approach

Machine learning is a natural approach — the robot can adapt to a given scenario, even if it was not programmed to handle it beforehand. Indeed, Deep Reinforcement Learning (deep RL), which has recently led to AI breakthroughs in computer games, has been publicized as the learning-based approach to robotics.

Current Limitations

To date, however, deep RL studies have focused on known and observable systems, where uncertainty was resolved by lengthy trial and error. Quickly learning to act in novel environments, as required for robotics, is not yet within our reach.

Challenges in Robotics

The crux of the matter is the tight coupling between perception and control under high uncertainty. The robot must actively reduce uncertainty while also trying to solve the task. For complex and high-dimensional systems, we do not have a suitable algorithmic framework for this.

Proposal Goals

In this proposal, our overarching goal is to:

1. Develop the algorithmic framework of using deep learning in problems that tightly couple perception, planning, and control.
2. Advance robotic AI to reliably manipulate general objects in unstructured environments.

Methodology

Towards this end, we shall:

• Develop neural network representations of uncertainty.
• Create algorithms that estimate uncertainty from data.
• Develop theory and algorithms for decision making under uncertainty, bringing in a fresh perspective to the problem based on Bayesian reinforcement learning (Bayes-RL).

Expected Outcomes

These advances will allow us to:

• Study safety certificates for deep RL.
• Develop a general and practical methodology for learning-based robotic manipulation under uncertainty, validated on real robot experiments.

Broader Impact

Aside from robotic manipulation, we expect impact on various fields where decision making plays an important role.