Skip to Main content Skip to Navigation
Conference papers

The Impact of Reward Shaping in Reinforcement Learning for Agent-based Microgrid Control

Abstract : In order to reduce CO2 emissions, electricity networks must increasingly integrate renewable energies. Microgrids are distributed electrical networks with their own generation and load, often supported by an electrical storage system. It can be connected to the external electrical network or isolated. Since electricity consumption, price and renewable production are stochastic phenomena, the control of microgrids must adapt to uncertainties. Data-driven models and in particular reinforcement learning (RL) have become efficient algorithms in high-level microgrid control. RL are agent-based algorithms, which interact with their environment and learn with a numerical reward signal. A certain behavior can implicitly be expected when the reward system is formulated. For example, a reward system that encourages the agent to interact as little as possible with the external network will explicitly increase the autonomy of the microgrid. Implicitly, it can be expected to schedule the battery to maximize the ratio of renewable energy used to the amount producible. Q-learning algorithm has been used due to its performance in discrete action space, which simplified the benchmark complexity. An agent is trained with different reward functions commonly found in the literature related to data-driven microgrid control algorithms. The agent parameters do not vary from one case study to another. Indicators are set up to evaluate the agent behavior. They are based on implicit behavioral criteria in the definition of the reward system such as the ratio of renewable energy used, the amount of energy stored during peak hours, etc. This study enables to find a way to rationalize the choice of a reward system to control in a near-optimal way microgrid while meeting implicit secondary objectives. It could lead to a choice on weighting coefficient in a combination of reward functions.
Document type :
Conference papers
Complete list of metadata

https://hal-mines-albi.archives-ouvertes.fr/hal-03754056
Contributor : IMT Mines Albi IMT Mines Albi Connect in order to contact the contributor
Submitted on : Friday, August 19, 2022 - 10:21:48 AM
Last modification on : Monday, August 22, 2022 - 9:25:16 AM

Identifiers

Citation

Valentin Père, Fabien Baillon, Mathieu Milhé, Jean-Louis Dirion. The Impact of Reward Shaping in Reinforcement Learning for Agent-based Microgrid Control. ESCAPE 32 - European Symposium on Computer Aided Porcess Engineering, Jun 2022, Toulouse, France. pp.1459-1464, ⟨10.1016/B978-0-323-95879-0.50244-7⟩. ⟨hal-03754056⟩

Share

Metrics

Record views

32