Reinforcement Learning Algorithms to Optimize the Integration of Electric Vehicle Services Into Power Systems

Electric Vehicles (EVs) present an opportunity to enhance the flexibility of power systems through their integration with Renewable Energy Sources (RES) via Vehicle-to-Grid and Vehicle-to-Home services. However, EVs integration introduce challenges related to their management, influenced by factors such as user behaviour, fluctuating RES generation, grid or building requirements and battery degradation. Within this framework, this paper explores the applications of Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) algorithms to solve optimization problems aimed at enhancing the flexibility services offered by EVs, maximizing the use of RES whenever possible, while accounting for battery degradation. Besides, limitations, potential solutions and new areas for further development are addressed. In particular, alternative solutions are proposed to address the challenge of requiring a large number of samples for the proper training of RL and DRL algorithms. Furthermore, to mitigate the models' dependence on stochastic variables-such as renewable energy production and load demand-the potential integration of forecasting models for these variables, as well as the implementation of virtual battery partitioning using RL and DRL algorithms is proposed.