A novel two-level deep reinforcement learning enabled game approach for incentive-based distributed voltage regulation with participation of autonomous photovoltaic inverters

Kang Xiong, Weihao Hu, Di Cao*, Guozhou Zhang, Zhe Chen, Frede Blaabjerg

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

The voltage violation issues caused by uncertainties of renewable energy generation severely restrict the operation of active distribution networks (ADN). Meanwhile, current voltage regulation methods using PV inverters always ignore the autonomy of PV prosumers. In this context, this paper proposes a novel two-level deep reinforcement learning (DRL) enabled incentive-based voltage regulation approach to mitigate voltage fluctuations while fully exploiting the potential of PV prosumers. To conduct voltage regulation in a distributed manner, the ADN is first partitioned into several sub-networks based on voltage-reactive sensitivity matrix. In the outer-level model, the distributed system operator (DSO) aims to select suitable compensation prices for different sub-networks to incentivize within prosumers to participate in voltage regulation, which is formulated as a Markov decision process (MDP) and solved by a centralized agent based soft actor-critic (SAC) algorithm. In the inner-level model, the autonomous prosumers within each sub-network feedback their corresponding voltage regulation strategies including both reactive power and active curtailment to maximize their own profits, which is modeled as Markov games and solved by a multi-agent SAC (MASAC) algorithm. To stabilize the training process and improve the computational efficiency, the inner-level agents are pre-trained and then the two-level agents are trained concurrently to solve the game problem. Case studies are conducted based on the IEEE 33-bus system, and the numerical results demonstrate that the proposed method can achieve effective performance on voltage regulation while guaranteeing prosumers’ profit.

Original languageEnglish
Article number135934
JournalEnergy
Volume324
ISSN0360-5442
DOIs
Publication statusPublished - 1 Jun 2025

Bibliographical note

Publisher Copyright:
© 2025 Elsevier Ltd

Keywords

  • Active distribution networks
  • Deep reinforcement learning
  • PV prosumers
  • Renewable energy
  • Voltage regulation

Fingerprint

Dive into the research topics of 'A novel two-level deep reinforcement learning enabled game approach for incentive-based distributed voltage regulation with participation of autonomous photovoltaic inverters'. Together they form a unique fingerprint.

Cite this