The Open-Source TEXPLORE Code Release for Reinforcement Learning on Robots

Abstract:

Reinforcement Learning (RL) is a paradigm for learning decision- making tasks that could enable cyber-physical systems (CPS) to learn and adapt to situations on-line. For an RL algorithm to be practical for CPS control tasks, it must learn in very few samples, while continually taking actions in real-time. In addition, the algorithm must learn effi- ciently in the face of noise, sensor/actuator delays and continuous state features. We describe TEXPLORE, a model-based RL method that ad- dresses these issues. It learns a random forest model of the domain which generalizes dynamics to unseen states. The agent targets exploration on states that are both promising for the final policy and uncertain in the model. With sample=based planning and a novel parallel architecture, TEXPLORE can select actions continually in real-time whenever neces- sary. TEXPLORE is the starting point of a newly begun CPS project that focuses on creating general learning algorithms for cyber-physical systems.