Reinforcement Learning Algorithms for CPS: TacTex'13- A Champion Adaptive Power Trading Agent

Abstract:

Sustainable energy systems of the future will no longer be able to rely on the current paradigm that energy supply follows demand. Many of the renewable energy resources do not produce power on demand, and therefore there is a need for new market structures that motivate sustainable behaviors by participants. The Power Trading Agent Competition (Power TAC) is a new annual competition that focuses on the design and operation of future retail power markets, specifically in smart grid environments with renewable energy production, smart metering, and autonomous agents acting on behalf of customers and retailers. It uses a rich, open-source simulation platform that is based on real-world data and state-of-the art customer models. Its purpose is to help researchers understand the dynamics of customer and retailer decision-making, as well as the robustness of proposed market designs. This paper introduces TACTEX'13, the champion agent from the inaugural competition in 2013. TACTEX'13 learns and adapts to the environment in which it operates, by heavily relying on reinforcement-learning and prediction methods. This paper describes the constituent components of TACTEX'13 and examines its success through analysis of competition results and subsequent controlled experiments.