REinforcement TWInning SysTems: from collaborative digital twins to model-based reinforcement learning

Introduction

The concept of digital twins promises to revolutionize engineering by offering new avenues for optimization, control, and predictive maintenance. Digital twins seek to virtually replicate systems using models that continuously "learn" from data, automatizing data collection, validation, and refinement and becoming "self-learning" models.

Current Challenges

However, this concept is not yet established in engineering and requires significant developments in integrating machine learning with traditional "domain-specific" knowledge.

Project Objectives

The Re-Twist project tackles this challenge with two objectives:

1. Framework Development
   The first objective is to develop a new framework that puts fundamental principles at the core of digital twinning. This framework combines the training of a digital twin with the training of a controlling agent in ways that allow one to learn from the other. The agent learns by trial and error, as in reinforcement learning, while interacting with the system and using the digital twin as a playground. This novel framework is called Reinforcement Twinning (RT).

2. Lab-Scale Prototypes
   The second objective is to develop RT on lab-scale prototypes of systems at the center of global challenges. These include:

   • Optimal operation of wind turbines
   • Drone propellers
   • Sloshing tanks
   • Cryogenic liquid storage

Importance of the Systems

Wind turbines drive the fastest-growing renewable energy sector. Drones have the potential to revolutionize monitoring, inspection, rescue missions, swift delivery of medical supplies, and more. The optimal management of cryogenic tanks, controlling the dynamics of sloshing and the thermodynamics of boil-off, will be essential to the economic viability of green fuels such as liquid hydrogen.

Risk and Potential Gains

This project is "high risk" because it endeavors to establish a new discipline at the intersection of machine learning and energy engineering. It promises "high gains" by aiming to experimentally validate twinning systems that could significantly impact society.