Formalised Reasoning about Expectations: Composable, Automated, Speedy, Trustworthy

Introduction

Automatic Differentiation (AD) systems, like TensorFlow, and probabilistic programming languages (PPLs), like Stan, automate complex computations of derivatives and Bayesian inference tasks. By streamlining these computations for non-expert users, these high-level systems have accelerated progress across science and society (e.g., by enabling machine learning).

Theoretical Foundations

Yet, the theoretical foundations needed to build a high-level system for composable programming with derivatives and probabilities are missing. This chasm in our knowledge severely limits the implementation of machine learning techniques, preventing them from reaching their full potential.

Key Challenges

Specifically, we do not understand:

1. How to perform AD on programs built using probabilistic choices and expected values.
2. How to compose (i.e., combine and integrate) Bayesian inference algorithms.

Project Overview

FoRECAST addresses this chasm by developing programming language theory and tools for flexible, composable, and efficient calculations with derivatives and probabilities.

Work Packages

• WP 1: Develops case studies in collaboration with domain experts to ensure that FoRECAST creates theory and systems relevant to real-world, complex modeling problems.

• WP 2: Develops the semantic foundations, algorithms, and formalized correctness proofs for composable AD of probabilistic programs.

• WP 3: Builds a practical stochastic (i.e., probabilistic) AD system that synthesizes these novel gradient estimation techniques.

• WP 4: Establishes theoretical foundations to compose Bayesian inference algorithms in PPLs.

• WP 5: Implements a user-friendly PPL that facilitates composable Bayesian inference, enabling more flexible modeling for a wider user base.

Conclusion

By mathematically formalizing, generalizing, optimizing, and implementing a next-generation PPL, this project will lay a trustworthy foundation upon which probabilistic data analysis applications (e.g., reinforcement learning, proteomics modeling, and paleoclimate reconstructions) can rise to the next level.