Hybrid and Interpretable Deep neural audio machines

Introduction

Machine Listening, or AI for Sound, is defined as the general field of Artificial Intelligence applied to audio analysis, understanding, and synthesis by a machine. The access to ever-increasing super-computing facilities, combined with the availability of huge data repositories (although largely unannotated), has led to the emergence of a significant trend with pure data-driven machine learning approaches.

Trends in Machine Listening

The field has rapidly moved towards end-to-end neural approaches which aim to directly solve the machine learning problem for raw acoustic signals. However, these approaches often only loosely take into account the nature and structure of the processed data.

Consequences of Current Approaches

The main consequences are that the models are:

1. Overly complex, requiring massive amounts of data to be trained and extreme computing power to be efficient (in terms of task performance).
2. Largely unexplainable and non-interpretable.

Proposed Solutions

To overcome these major shortcomings, we believe that our prior knowledge about the nature of the processed data, their generation process, and their perception by humans should be explicitly exploited in neural-based machine learning frameworks.

Project Aim

The aim of HI-Audio is to build such hybrid deep approaches combining:

• Parameter-efficient and interpretable signal models
• Musicological and physics-based models
• Highly tailored, deep neural architectures

Research Directions

The research directions pursued in HI-Audio will exploit novel deterministic and statistical audio and sound environment models with dedicated neural auto-encoders and generative networks. The project will target specific applications including:

• Speech and audio scene analysis
• Music information retrieval
• Sound transformation and synthesis