Classical control approaches are based on physical dynamic models, which are required to describe the true underlying system behaviour in a sufficiently accurate fashion. For complex dynamical systems, however, such descriptions are often extremely hard to obtain or even nonexistent, hence data-driven approaches have to be employed. Data-driven models are based on observations and measurements of the true system and only require a minimum amount of prior knowledge of the system. However, they require new control approaches since classic analysis and synthesis tools are not suitable for models of probabilistic nature. Our work focuses on the Gaussian process model, which is very generally applicable and has shown to be successful in many control scenarios. We develop new control algorithms, which not only improve the overall performance but also guarantee the stability of the closed-loop system. Finally, the approaches are tested and validated in robotic experiments.
Identification and Control with Gaussian Processes
Data-driven approaches from machine learning provide powerful tools to identify dynamical systems with limited prior knowledge of the model structure. These are, on the one side, very flexible to model a large variety of systems, but, on the other side, also bring new challenges: Classical control approaches need to be adapted to work successfully on data-based models, certain desired properties on convergence are difficult to prove and multiple ways exist to exploit the prior knowledge available.
- How to enforce stability in data-driven models?
- Which control laws allow formal guarantees in closed-loop systems?
- Can knowledge on model fidelity be used in the control design?
On the system identification side, we focus on Gaussian processes to model unknown systems. We use approaches from robust and adaptive control in the design and analysis of the controller to handle imprecision in the identified model. Since data-driven models are often of probabilistic nature, tools for stochastic differential equations are required. For the future, we are planning to apply stochastic optimal control and scenario-based model predictive control in this setting.
- Different methods for the identification of systems which are a priori known to be stable have been developed.
- A closed-loop identification of control-affine systems using Gaussian processes was proposed and applied in a feedback linearization setting.
- Developing of a GPR-based control law for Lagrangian systems which guarantees a bounded tracking error of the closed-loop system.
- Stability properties of Gaussian Process State Space Models for different kernel functions.
Optimal Learning Control based on Gaussian Processes
Researcher: Armin Lederer
Model predictive control is a modern control technique that has been applied to a wide variety of systems. Its success stems from its capability to explicitly handle constraints on states and control inputs as well as simple implementation of tracking control. However, it requires a precise model of the controlled system, which is often not available because of high system complexity or inherent system uncertainty. Gaussian processes offer a solution to this issue by allowing to learn system models from data of the system dynamics. Nevertheless, using learned models in model predictive control raises questions at the intersection between machine learning in control theory.
- Can Gaussian processes be modified to allow on-line learning while providing theoretical learning error bounds?
- Is it possible to guarantee stability of a system controlled by model predictive control if only samples of the system's dynamics are known?
- How can model uncertainty be exploited for control as well as learning in closed-loop?
- Do Gaussian processes exist which facilitate the design of optimal control?
For on-line learning we focus on local Gaussian processes and Gaussian processes with compactly supported kernels allowing exact inference. By combining these approaches with methods from computational geometry they can be implemented efficiently. On the control side we apply sampling based approaches for stability verification and parameter optimization of model predictive control. Furthermore, we develop robust control strategies tailored to the setting provided by Gaussian process models. In the future we plan to investigate the effect of on-line learning on closed-loop stability and to develop control schemes that allow optimal learning in closed-loop.