Deep reinforcement learning for multiple reservoir operation planning in the Chao Phraya River Basin

Abstract

This study demonstrates application of Deep Deterministic Policy Gradient (DDPG)-based algorithm to provide comprehensive and flexible plans for reservoir operation planning of the multiple reservoir system in the Chao Phraya River Basin (CPYRB), Thailand aiming to mitigate flood and drought risks in the region. The multi-agent-based Deep Reinforcement Learning (DRL) model is accordingly constructed considering 7-D predicted inflow, reservoir water released from adjacent reservoir, downstream flow condition, and changes in reservoir water storage, as state variables. The desired goal is to increase water storage levels in all reservoirs by 10–15% to ensure higher potential in supplying water for crop cultivation over the dry seasons and preventing flood occurrences during wet season. Simulation results from 2009 to 2022 indicate that DRL–DDPG-based algorithm can perform well in solving sequential decision problems for optimal operation of multiple reservoir system to achieve the desired water storage goal. It can offer realistic simulation results of seasonal and annual release schemes and reservoir release ratios among reservoirs in the system compared to actual operation and Fmincon and ANFIS optimizations. Importantly, DRL model demonstrates a significant advantage in view of increasing the long-term water storage levels in all reservoirs as targeted in the modelling process while maintaining the similar and consistent release schemes in the reservoir system. For the multipurpose multiple reservoir system operation, adjusting the dynamic desired goals within multi-agent-based RL model is advisable to attain the specific desired outcomes and address various water scenarios.