Bidding strategy for trading wind energy and purchasing reserve of wind power producer–A DRL based approach

Wind power producers (WPP) are punished when take part in the short-term electricity market due to the inaccuracy of wind power prediction. The profit loss can be partially offset by strategic reserve purchasing. Due to the uncertainty of real-time wind power production, the reserve capacity price and the highly dynamic price regulation, it is difficult to determine the quantity of reserve to purchase, which might have a great impact on the profit of WPP. This paper investigates the possible influence on the revenue of WPP if they take part in both the energy and reserve market. This problem is first formulated as a Markov decision process (MDP). After that, the asynchronous advantage actor-critic (A3C) method is used to take care of this problem. Several agents are applied to explore the action space simultaneously. Neural networks are applied to fit the policy function and value function, which are trained against each other to learn the optimal dynamic bidding strategy from historical data by constantly trial and error. Simulation results of a wind farm in Denmark validate that the profit loss of the WPP can be significantly reduced when the A3C algorithm is used for bidding strategy formulation when WPP participate in both the energy and reserve market.