Reconciling Classical and Modern (Deep) Machine Learning for Real-World Applications

Introduction

Despite the undeniable success of machine learning in addressing a wide variety of technological and scientific challenges, the current trend of training predictive models with an ever-growing number of parameters from an ever-growing amount of data is not sustainable.

Challenges of Current Models

These huge models, often engineered by large corporations benefiting from substantial computational resources, typically require learning a billion or more parameters. They have proven to be very effective in solving prediction tasks in various fields, including:

• Computer vision
• Natural language processing
• Computational biology

However, they mostly remain black boxes that are hard to interpret, computationally demanding, and not robust to small data perturbations.

Objectives of APHELEIA

With a strong emphasis on visual modeling, the grand challenge of APHELEIA is to develop a new generation of machine learning models that are:

1. More robust
2. Interpretable
3. Efficient
4. Not reliant on massive amounts of data to produce accurate predictions

To achieve this objective, we will foster new interactions between:

• Classical signal processing
• Statistics
• Optimization
• Modern deep learning

Approach to Model Development

Our goal is to reduce the need for massive data by enabling scientists and engineers to design trainable machine learning models that:

• Directly encode a priori knowledge of the task semantics and data formation process
• Automatically prefer simple and stable solutions over complex ones

These models will be built on solid theoretical foundations with convergence and robustness guarantees, which are important for making real-life trustworthy predictions in the wild.

Implementation and Impact

We will implement these ideas in an open-source software toolbox readily applicable to visual recognition and inverse imaging problems, which will also handle other modalities. This will stimulate interdisciplinary collaborations, with the potential to be a game changer in the way scientists and engineers design machine learning problems.