The missing mathematical story of Bayesian uncertainty quantification for big data

Introduction

Recent years have seen a rapid increase in available information. This has created an urgent need for fast statistical and machine learning methods that can scale up to big data sets.

Challenges with Standard Approaches

Standard approaches, including the now routinely used Bayesian methods, are becoming computationally infeasible, especially in complex models with many parameters and large data sizes. A variety of algorithms have been proposed to speed up these procedures, but these are typically black box methods with very limited theoretical support.

Concerns in Real-World Applications

In fact, empirical evidence shows the potentially bad performance of such methods. This is especially concerning in real-world applications, e.g., in medicine.

Project Objectives

In this project, I shall open up the black box and provide a theory for scalable Bayesian methods combining recent, state-of-the-art techniques from:

1. Bayesian nonparametrics
2. Empirical process theory
3. Machine learning

Focus Areas

I focus on two very important classes of scalable techniques:

• Variational Bayes
• Distributed Bayes

Establishing Guarantees and Limitations

I shall establish guarantees, but also limitations, of these procedures for:

• Estimating the parameter of interest
• Quantifying the corresponding uncertainty

This will be done within a framework that will also convince those outside of the Bayesian paradigm.

Expected Outcomes

As a result, scalable Bayesian techniques will have more accurate performance and also better acceptance by a wider community of scientists and practitioners.

Nature of the Research

The proposed research, although motivated by real-world problems, is of a mathematical nature. In the analysis, I consider mathematical models, which are routinely used in various fields (e.g., high-dimensional linear and logistic regressions are the workhorses in econometrics or genetics).

Practical Applications

My theoretical results will provide principled new insights that can be used, for instance, in multiple specific applications I am involved in, including:

1. Developing novel statistical methods for understanding fundamental questions in cosmology
2. Early detection of dementia using multiple data sources.