Certifying Black-Box Policies With Stability for Nonlinear Control

Certifying Black-Box Policies With Stability for Nonlinear Control

Blog Article

Machine-learned black-box policies are ubiquitous for nonlinear control problems.Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics.

We study the problem of certifying a black-box control policy with stability using model-based advice for nonlinear control on a single trajectory.We first show a general negative result that Foggers a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing.We then propose an adaptive $lambda$-confident policy, with a coefficient $lambda$ indicating the confidence in a black-box policy, and prove its stability.With bounded nonlinearity, in addition, we show that the adaptive $lambda$-confident policy achieves a bounded competitive ratio when a black-box policy Colander is near-optimal.

Finally, we propose an online learning approach to implement the adaptive $lambda$-confident policy and verify its efficacy in case studies about the Cart-Pole problem and a real-world electric vehicle (EV) charging problem with covariate shift due to COVID-19.