TY - JOUR
T1 - A novel two-level deep reinforcement learning enabled game approach for incentive-based distributed voltage regulation with participation of autonomous photovoltaic inverters
AU - Xiong, Kang
AU - Hu, Weihao
AU - Cao, Di
AU - Zhang, Guozhou
AU - Chen, Zhe
AU - Blaabjerg, Frede
N1 - Publisher Copyright:
© 2025 Elsevier Ltd
PY - 2025/6/1
Y1 - 2025/6/1
N2 - The voltage violation issues caused by uncertainties of renewable energy generation severely restrict the operation of active distribution networks (ADN). Meanwhile, current voltage regulation methods using PV inverters always ignore the autonomy of PV prosumers. In this context, this paper proposes a novel two-level deep reinforcement learning (DRL) enabled incentive-based voltage regulation approach to mitigate voltage fluctuations while fully exploiting the potential of PV prosumers. To conduct voltage regulation in a distributed manner, the ADN is first partitioned into several sub-networks based on voltage-reactive sensitivity matrix. In the outer-level model, the distributed system operator (DSO) aims to select suitable compensation prices for different sub-networks to incentivize within prosumers to participate in voltage regulation, which is formulated as a Markov decision process (MDP) and solved by a centralized agent based soft actor-critic (SAC) algorithm. In the inner-level model, the autonomous prosumers within each sub-network feedback their corresponding voltage regulation strategies including both reactive power and active curtailment to maximize their own profits, which is modeled as Markov games and solved by a multi-agent SAC (MASAC) algorithm. To stabilize the training process and improve the computational efficiency, the inner-level agents are pre-trained and then the two-level agents are trained concurrently to solve the game problem. Case studies are conducted based on the IEEE 33-bus system, and the numerical results demonstrate that the proposed method can achieve effective performance on voltage regulation while guaranteeing prosumers’ profit.
AB - The voltage violation issues caused by uncertainties of renewable energy generation severely restrict the operation of active distribution networks (ADN). Meanwhile, current voltage regulation methods using PV inverters always ignore the autonomy of PV prosumers. In this context, this paper proposes a novel two-level deep reinforcement learning (DRL) enabled incentive-based voltage regulation approach to mitigate voltage fluctuations while fully exploiting the potential of PV prosumers. To conduct voltage regulation in a distributed manner, the ADN is first partitioned into several sub-networks based on voltage-reactive sensitivity matrix. In the outer-level model, the distributed system operator (DSO) aims to select suitable compensation prices for different sub-networks to incentivize within prosumers to participate in voltage regulation, which is formulated as a Markov decision process (MDP) and solved by a centralized agent based soft actor-critic (SAC) algorithm. In the inner-level model, the autonomous prosumers within each sub-network feedback their corresponding voltage regulation strategies including both reactive power and active curtailment to maximize their own profits, which is modeled as Markov games and solved by a multi-agent SAC (MASAC) algorithm. To stabilize the training process and improve the computational efficiency, the inner-level agents are pre-trained and then the two-level agents are trained concurrently to solve the game problem. Case studies are conducted based on the IEEE 33-bus system, and the numerical results demonstrate that the proposed method can achieve effective performance on voltage regulation while guaranteeing prosumers’ profit.
KW - Active distribution networks
KW - Deep reinforcement learning
KW - PV prosumers
KW - Renewable energy
KW - Voltage regulation
UR - http://www.scopus.com/inward/record.url?scp=105002228977&partnerID=8YFLogxK
U2 - 10.1016/j.energy.2025.135934
DO - 10.1016/j.energy.2025.135934
M3 - Journal article
AN - SCOPUS:105002228977
SN - 0360-5442
VL - 324
JO - Energy
JF - Energy
M1 - 135934
ER -