Bert Kappen (Radboud University)
Abstract. Intelligent systems, whether natural or artificial, must act in a world that is highly unpredictable. To plan actions with uncertainty is a stochastic optimal control problem. However, there are two fundamental problems: the optimal control solution is intractable to compute and intractable to represent due the non-trivial state dependence of the optimal control. This has prevented large scale application of stochastic optimal control theory sofar. The path integral control theory describes a class of control problems whose solution can be computed as an inference computation. In this presentation we formalize the intuitive notion that the efficiency of the inference computation is related to the proximity of the sampling control to the optimal control. Secondly, we show new results that allow approximate computation of state dependent optimal controls using the cross entropy method. These two ingredients together suggest a novel adaptive sampling procedure, called PICE (path integral cross entropy method), that learns a controller based on self-generated data. |The adaptive sampling procedure can be used to efficiently compute optimal controls but can also be used to accelerate other Monte Carlo computations. We illustrate the results on a few examples in robotics and time series.