Adaptive dynamic planning control method and system for energy storage station, and storage medium

11326579 · 2022-05-10

Assignee

Inventors

Cpc classification

International classification

Abstract

An adaptive dynamic planning control method and system for a large-scale energy storage station. The method comprises: setting a structure and control target parameters of an adaptive dynamic planning control system; initializing the parameters and importing an initial state of a controlled object; calculating an original wind electricity power fluctuation rate at a current moment t and smoothing the original wind electricity power according to a change rate control strategy; calculating a smoothed wind storage power fluctuation rate, a power of an energy storage system, and a state of charge (SOC) of the energy storage system; initializing and training an evaluation module and an execution module; calculating and saving a control strategy, a smoothed wind storage power fluctuation rate, an energy storage power and a (SOC) at each moment; and outputting the control strategy at each moment, the smoothed wind storage power fluctuation rate, the energy storage power and the (SOC).

Claims

1. A method for Adaptive Dynamic Programming (ADP) control by a power station for large-scale energy storage, comprising: setting an objective control parameter and a structure of a system for ADP control; performing parameter initialization, and importing an initialized parameter as an initial state of a controlled object; for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage; performing initialization for training an estimation module and a management module; for each time point, computing and storing a control strategy, the rate of turbulence of the wind and energy storage hybrid power smoothed, power of energy storage, and the SOC of the system for energy storage; and outputting the rate of turbulence of the wind and energy storage hybrid power smoothed, the power of energy storage, the SOC of the system for energy storage, and the control strategy for the each time point, wherein the system for ADP control comprises a two-layer structure of the estimation module and the management module, wherein each of the estimation module and the management module is built with a three-layer structure of a neural network, wherein the objective control parameter comprises a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of the system for energy storage, a limiting range of the SOC of the system for energy storage, a sampling interval Δt, an observation time T, an objective rate of turbulence r.sub.obj.sup.T, and a limiting rate of turbulence r.sub.lim.sup.T.

2. The method according to claim 1, wherein the initialized parameter comprises an initial rate of turbulence of wind power, an initial SOC of the system for energy storage, and actual wind power output at the present time point.

3. The method according to claim 1, wherein the for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power hybrid r.sub.hybrid.sup.T smoothed, power P.sub.PBESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage comprises: for the present time point t, computing the rate of turbulence of the raw wind power r.sub.wp.sup.T using formulae of r wp T = f wp ( P wp max - P wp min P wp rated ) , ( 1 ) P wp max = max { P wp ( t ) , P wp ( t - Δ t ) , .Math. , P wp ( t - ( n - 1 ) Δ t ) } , ( 2 ) P wp min = min { P wp ( t ) , P wp ( t - Δ t ) , .Math. , P wp ( t - ( n - 1 ) Δ t ) } , and ( 3 ) T = n Δ t , ( 4 ) wherein the P.sub.wp.sup.rated is a capacity of wind power, i.e., rated power, the P.sub.wp.sup.max is a maximal wind power sampled within an observation time T, the P.sub.wp(t) is a minimal wind power sampled within the observation time, the Δt is a sampling interval, the P.sub.wp(t) is the raw wind power, the f.sub.up is a raw function for computing a rate of turbulence of the wind power, and the n is a number of sampling points within the observation time; smoothing the raw wind power by controlling the rate of change by defining the rate of change k(t) of the wind power according to the raw wind power P.sub.wp(t), a smoothed wind power P.sub.hybrid(t), and the sampling interval Δt according to a formula of k ( t ) = P wp ( t ) - P hybrid ( t - Δ t ) Δ t , ( 5 ) and controlling a rate of turbulence of wind and energy storage hybrid power to be within a required range using the control strategy comprising that: for k.sub.hybrid.sup.drop≤k(t)≤k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)  (6), for k(t)>k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.rise  (7), and for k(t)<k.sub.hybrid.sup.drop,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.drop  (8), wherein the k.sub.hybrid.sup.rise is a limiting rate of change that limits rise of wind power output, and the k.sub.hybrid.sup.drop is a limiting rate of change that limits drop of the wind power output, with k hybrid rise = P wp rated × r obj T T , and ( 9 ) k hybrid drop = - P wp rated × r obj T T , ( 10 ) wherein the r.sub.obj.sup.T is an objective rate of turbulence; computing the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed by controlling the rate of change, by using formulae of r hybrid T = f hybrid ( P hybrid max - P hybrid min P hybrid rated ) , ( 11 ) P hybrid max = max { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } , ( 12 ) P hybrid min = min { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } , and ( 13 ) T = n Δ t , ( 14 ) wherein the P.sub.hybrid.sup.max is a maximal wind and energy storage hybrid power sampled within the observation time T, and the P.sub.hybrid.sup.min is a minimal wind and energy storage hybrid power sampled within the observation time, wherein an objective of controlling a rate of turbulence of power using the change rage is to control the rate of turbulence of the wind and energy storage hybrid power smoothed to be less than a specified objective rate within the observation time, as shown by a formula of
r.sub.hybrid.sup.T<r.sub.obj.sup.T  (15), wherein the r.sub.obj.sup.T is the objective rate of turbulence of the wind and energy storage hybrid power within the observation time T; computing the power of the system for energy storage as
P.sub.BESS(t)=P.sub.hybrid(t)−P.sub.wp(t)  (16); and computing the SOC of the energy storage as follows, wherein for P.sub.BESS(t)>0, the system for energy storage is discharged, and the SOC decreases as shown in a formula of SOC ( t ) = SOC ( t - Δ t ) - Δ t .Math. P BESS ( t ) .Math. W bat , ( 17 ) wherein for P.sub.BESS(t)<0, the system for energy storage is charged, and the SOC increases as shown in a formula of SOC ( t ) = SOC ( t - Δ t ) + Δ t .Math. P BESS ( t ) .Math. W bat , ( 18 ) wherein the W.sub.bat is a capacity of the system for energy storage.

4. The method according to claim 1, wherein the performing initialization for training an estimation module and a management module comprises: initializing a discount factor α, a learning rate l.sub.a of the management module, a learning rate l.sub.c of the estimation module, a weight W.sub.a of the management module, a weight W.sub.c of the estimation module, a maximal number of cycles, and an expected error of module training, by setting an initial weight as a random number within (−1, +1), and selecting and adjusting the learning rates and the discount factor according to an objective result of control.

5. The method according to claim 1, wherein the for each time point, computing and storing a control strategy, the rate of turbulence of the wind and energy storage hybrid power smoothed, power of energy storage, and the SOC of the system for energy storage comprises steps of: determining whether the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T meets r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T; in response to determining that it does, taking, by the system for energy storage, no action to correct the power of energy storage; otherwise in response to determining that it does not, performing a next step of ADP module training in seeking an optimal correction to the power of energy storage; training the estimation module by inputting the control strategy and a state of the controlled object to the estimation module, updating the weight of the estimation module, and outputting a cost function; training the management module by inputting, to the management module, the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the power of energy storage P.sub.BESS(t), updating the weight of the management module, and outputting the control strategy as a correction ΔP.sub.BESS(t) to the power of energy storage; and storing the control strategy for the each time point, computing the state of the controlled object for a next time point t=t+1 and repeating the steps until control ends.

6. The method according to claim 5, wherein the determining whether the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T meets r.sub.lim.sup.T≤r.sub.hybrid.sup.T r.sub.obj.sup.T comprises: in response to determining that 0<r.sub.hybrid.sup.T<r.sub.lim.sup.T, determining that the system for energy storage has been outputting too much power and needs reverse correction of
P.sub.BESS.sup.ADPs(t)=P.sub.BESS(t)+ΔP.sub.BESS(t)  (22), wherein the ΔP.sub.BESS(t) is the correction to the power of energy storage; and in response to determining that r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T, determining that the system for energy storage has been outputting proper power and needs no correction, i.e.,
P.sub.BESS.sup.ADP(t)=P.sub.BESS(t)  (23), wherein the P.sub.BESS.sup.ADP(t) is the power of the system for energy storage acquired by ADP of the power of energy storage by controlling the rate of change.

7. The method according to claim 5, wherein the training the estimation module by inputting the control strategy and a state of the controlled object to the estimation module, updating the weight of the estimation module, and outputting a cost function comprises: normalizing the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, to be within [−1, +1]; inputting the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, to the estimation module, computing the cost function J.sub.c.sup.WPBESS(t) output by the estimation module, training the estimation module by building an objective function E.sub.chybrid(t), and updating a weight of a neural network of the estimation module according to a weight updating formula of the estimation module, using formulae of
U(t)=[r.sub.hybrid.sup.T,ΔP.sub.BESS(t),t]  (24),
J.sub.chybrid(t)=J.sub.c[r.sub.hybrid.sup.T,ΔP.sub.BESS(t),t,W.sub.c]  (25), and E chybrid ( t ) = 1 2 [ J chybrid ( t ) - U ( t + 1 ) - β c J chybrid ( t + 1 ) ] 2 , ( 26 ) wherein the cost function J.sub.chybrid(t) is the output of the estimation module, the U(t) is a utility function of r.sub.hybrid.sup.T, ΔP.sub.BESS(t), t defined according to an objective result of control, and the β.sub.c is a discount factor; and updating the weight W.sub.c of the neural network of the estimation module by training the estimation module via gradient descent or particle swarm optimization to minimize the objective function E.sub.chybrid (t), and ending the training in response to determining that the objective function E.sub.chybrid (t) has decreased to a set error or a maximal number of iterations has been reached.

8. The method according to claim 5, wherein the training the management module by inputting, to the management module, the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the power of energy storage P.sub.BESS(t), updating the weight of the management module, and outputting the control strategy as a correction ΔP.sub.BESS(t) BESS to the power of energy storage comprises: training the management module by inputting, to the management module, the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the power of energy storage P.sub.BESS(t); adjusting the control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, by minimizing an output J.sub.chybrid(t) of the estimation module using formulae of
ΔP.sub.BESS(t)=u[r.sub.hybrid.sup.T,t,W.sub.a]  (27), and
E.sub.ahybrid(t)=½[J.sub.chybrid(t)].sup.2  (28), wherein the control strategy ΔP.sub.BESS(t) is output by the management module to adjust the power of energy storage to vary within a proper range to reduce a range of turbulence of the SOC of the system for energy storage, and the u indicates that the control strategy ΔP.sub.BESS(t) is a function of r.sub.hybrid.sup.T, W.sub.a; and updating a weight W.sub.a of a neural network of the management module by training the management module via gradient descent or particle swarm optimization to minimize an objective function E.sub.ahybrid(t), and ending the training in response to determining that the objective function E.sub.ahybrid(t) has decreased to a set error or a maximal number of iterations has been reached.

9. A system for Adaptive Dynamic Programming (ADP) control by a power station for large-scale energy storage, comprising: a processor; and memory storing instructions executable by the processor, wherein the processor is arranged for: setting an objective control parameter and a structure of a system for ADP control; performing parameter initialization, and importing an initialized parameter as an initial state of a controlled object; for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage; performing initialization for training an estimation module and a management module; for each time point, computing and storing a control strategy, the rate of turbulence of the wind and energy storage hybrid power smoothed, power of energy storage, and the SOC of the system for energy storage; and outputting the rate of turbulence of the wind and energy storage hybrid power smoothed, the power of energy storage, the SOC of the system for energy storage, and the control strategy for the each time point, wherein the system for ADP control comprises a two-layer structure of the estimation module and the management module, wherein each of the estimation module and the management module is built with a three-layer structure of a neural network, wherein the objective control parameter comprises a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of the system for energy storage, a limiting range of the SOC of the system for energy storage, a sampling interval Δt , an observation time T, an objective rate of turbulence r.sub.obj.sup.T, and a limiting rate of turbulence r.sub.lim.sup.T.

10. The system according to claim 9, wherein the system for ADP control comprises a two-layer structure of the estimation module and the management module, wherein each of the estimation module and the management module is built with a three-layer structure of a neural network, wherein the objective control parameter comprises a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of the system for energy storage, a limiting range of the SOC of the system for energy storage, a sampling interval Δt, an observation time T, an objective rate of turbulence r.sub.obj.sup.T, and a limiting rate of turbulence.

11. The system according to claim 9, wherein the initialized parameter comprises an initial rate of turbulence of wind power, an initial SOC of the system for energy storage, and actual wind power output at the present time point.

12. The system according to claim 9, wherein the for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage comprises: for the present time point t, computing the rate of turbulence of the raw wind power r.sub.wp.sup.T using formulae of r wp T = f wp ( P wp max - P wp min P wp rated ) , ( 1 ) P wp max = max { P wp ( t ) , P wp ( t - Δ t ) , .Math. , P wp ( t - ( n - 1 ) Δ t ) } , ( 2 ) P wp min = min { P wp ( t ) , P wp ( t - Δ t ) , .Math. , P wp ( t - ( n - 1 ) Δ t ) } , and ( 3 ) T = n Δ t , ( 4 ) wherein the P.sub.wp.sup.rated is a capacity of wind power, i.e., rated power, the P.sub.wp.sup.max is a maximal wind power sampled within an observation time T, the P.sub.wp.sup.min is a minimal wind power sampled within the observation time, the Δt is a sampling interval, the P.sub.wp(t) is the raw wind power, the f.sub.wp is a raw function for computing a rate of turbulence of the wind power, and the n is a number of sampling points within the observation time; smoothing the raw wind power by controlling the rate of change by defining the rate of change k(t) of the wind power according to the raw wind power P.sub.wp(t), a smoothed wind power P.sub.hybrid(t), and the sampling interval Δt according to a formula of k ( t ) = P wp ( t ) - P hybrid ( t - Δ t ) Δ t , ( 5 ) and controlling a rate of turbulence of wind and energy storage hybrid power to be within a required range using the control strategy comprising that: for k.sub.hybrid.sup.drop≤(t)≤k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)  (6), for k(t)>k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.rise  (7), and for k(t)<k.sub.hybrid.sup.drop,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.drop  (8), wherein the k.sub.hybrid.sup.rise is a limiting rate of change that limits rise of wind power output, and the k.sub.hybrid.sup.drop is a limiting rate of change that limits drop of the wind power output, with k hybrid rise = P wp rated × r obj T T , and ( 9 ) k hybrid drop = - P wp rated × r obj T T , ( 10 ) wherein the r.sub.obj.sup.T is an objective rate of turbulence; computing the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed by controlling the rate of change, by using formulae of r hybrid T = f hybrid ( P hybrid max - P hybrid min P hybrid rated ) , ( 11 ) P hybrid max = max { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } , ( 12 ) P hybrid min = min { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } , and ( 13 ) T = n Δ t , ( 14 ) wherein the P.sub.hybrid.sup.max is a maximal wind and energy storage hybrid power sampled within the observation time T, and the P.sub.hybrid.sup.min is a minimal wind and energy storage hybrid power sampled within the observation time, wherein an objective of controlling a rate of turbulence of power using the change rage is to control the rate of turbulence of the wind and energy storage hybrid power smoothed to be less than a specified objective rate within the observation time, as shown by a formula of
r.sub.hybrid.sup.T<r.sub.obj.sup.T  (15), wherein the r.sub.obj.sup.T is the objective rate of turbulence of the wind and energy storage hybrid power within the observation time T; computing the power of the system for energy storage as
P.sub.BESS(t)=P.sub.hybrid(t)−P.sub.wp(t)  (16); and computing the SOC of the energy storage as follows, wherein for P.sub.BESS(t)>0, the system for energy storage is discharged, and the SOC decreases as shown in a formula of SOC ( t ) = SOC ( t - Δ t ) - Δ t .Math. P BESS ( t ) .Math. W bat , ( 17 ) wherein for P.sub.BESS(t)<0, the system for energy storage is charged, and the SOC increases as shown in a formula of SOC ( t ) = SOC ( t - Δ t ) + Δ t .Math. P BESS ( t ) .Math. W bat , ( 18 ) wherein the W.sub.bat is a capacity of the system for energy storage.

13. The system according to claim 9, wherein the performing initialization for training an estimation module and a management module comprises: initializing a discount factor α, a learning rate l.sub.a of the management module, a learning rate l.sub.c of the estimation module, a weight W.sub.a of the management module, a weight W.sub.c of the estimation module, a maximal number of cycles, and an expected error of module training, by setting an initial weight as a random number within (−1, +1), and selecting and adjusting the learning rates and the discount factor according to an objective result of control.

14. The system according to claim 9, wherein the for each time point, computing and storing a control strategy, the rate of turbulence of the wind and energy storage hybrid power smoothed, power of energy storage, and the SOC of the system for energy storage comprises steps of: determining whether the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T meets r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T; in response to determining that it does, taking, by the system for energy storage, no action to correct the power of energy storage; otherwise in response to determining that it does not, performing a next step of ADP module training in seeking an optimal correction to the power of energy storage; training the estimation module by inputting the control strategy and a state of the controlled object to the estimation module, updating the weight of the estimation module, and outputting a cost function; training the management module by inputting, to the management module, the state of the controlled object comprising the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T and the power of energy storage P.sub.BESS(t), updating the weight of the management module, and outputting the control strategy as a correction ΔP.sub.BESS(t) to the power of energy storage; and storing the control strategy for the each time point, computing the state of the controlled object for a next time point t=t+1 and repeating the steps until control ends.

15. The system according to claim 14, wherein the determining whether the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T meets r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T comprises: in response to determining that 0<r.sub.hybrid.sup.T<r.sub.lim.sup.T, determining that the system for energy storage has been outputting too much power and needs reverse correction of
P.sub.BESS.sup.ADP(t)=P.sub.BESS(t)+ΔP.sub.BESS(t)  (22), wherein the ΔP.sub.BESS(t) is the correction to the power of energy storage; and in response to determining that r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T, determining that the system for energy storage has been outputting proper power and needs no correction, i.e.,
P.sub.BESS.sup.ADP(t)=P.sub.BESS(t)  (23), wherein the P.sub.BESS.sup.ADP(t) is the power of the system for energy storage acquired by ADP of the power of energy storage by controlling the rate of change.

16. A method for Adaptive Dynamic Programming (ADP) control by a power station for large-scale energy storage, comprising: for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage; determining whether the r.sub.hybrid.sup.T, the P.sub.BESS(t), and the SOC are within constraint ranges corresponding to an objective control parameter; in response to determining that they are not within the constraint ranges, inputting the r.sub.hybrid.sup.T to the management module for the present time point; for the present time point, outputting, by the management module according to the r.sub.hybrid.sup.T, a control strategy for controlling charging power and discharging power of the system for energy storage of the power station for large-scale energy storage; inputting the r.sub.hybrid.sup.T and the control strategy for the present time point to the estimation module for the present time point; outputting, by the estimation module, a cost function according to the r.sub.hybrid.sup.T and the control strategy for the present time point; building, according to the r.sub.hybrid.sup.T and the cost function, an objective function for training the estimation module; training the estimation module according to the objective function to minimize the objective function; and training the management module according to the cost function to minimize the cost function, wherein the estimation module and the management module are retrained for outputting the control strategy for a next time point.

17. The method according to claim 16, wherein the control strategy comprises a correction to power of energy storage.

18. A non-tranistory computer-readable storage medium having stored therein instructions executable by a computer to perform a method for Adaptive Dynamic Programming (ADP) control by a power station for large-scale energy storage, the method comprising: setting an objective control parameter and a structure of a system for ADP control; performing parameter initialization, and importing an initialized parameter as an initial state of a controlled object; for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage; performing initialization for training an estimation module and a management module; for each time point, computing and storing a control strategy, the rate of turbulence of the wind and energy storage hybrid power smoothed, power of energy storage, and the SOC of the system for energy storage; and outputting the rate of turbulence of the wind and energy storage hybrid power smoothed, the power of energy storage, the SOC of the system for energy storage, and the control strategy for the each time point, wherein the system for ADP control comprises a two-layer structure of the estimation module and the management module, wherein each of the estimation module and the management module is built with a three-layer structure of a neural network, wherein the objective control parameter comprises a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of the system for energy storage, a limiting range of the SOC of the system for energy storage, a sampling interval Δt, an observation time T, an objective rate of turbulence r.sub.obj.sup.T, and a limiting rate of turbulence r.sub.lim.sup.T.

19. A non-transitory computer-readable storage medium having stored therein instructions executable by a computer to perform a method for Adaptive Dynamic Programming (ADP) control by a power station for large-scale energy storage, the method comprising: for a present time point t, computing a rate of turbulence of raw wind power r.sub.wp.sup.T, smoothing the raw wind power by controlling a rate of change, and computing a rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed, power P.sub.BESS(t) of a system for energy storage, and a State Of Charge (SOC) of the system for energy storage; determining whether the r.sub.hybrid.sup.T, P.sub.BESS(t), and the SOC are within constraint ranges corresponding to an objective control parameter; in response to determining that they are not within the constraint ranges, inputting the r.sub.hybrid.sup.T to the management module for the present time point; for the present time point, outputting, by the management module according to the r.sub.hybrid.sup.T, a control strategy for controlling charging power and discharging power of the system for energy storage of the power station for large-scale energy storage; inputting the r.sub.hybrid.sup.T and the control strategy for the present time point to the estimation module for the present time point; outputting, by the estimation module, a cost function according to the r.sub.hybrid.sup.T and the control strategy for the present time point; building, according to the r.sub.hybrid.sup.T and the cost function, an objective function for training the estimation module; training the estimation module according to the objective function to minimize the objective function; and training the management module according to the cost function to minimize the cost function, wherein the estimation module and the management module are retrained for outputting the control strategy for a next time point.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a diagram of a structure of a system for ADP control by a power station for large-scale energy storage according to an embodiment herein.

(2) FIG. 2 is a flowchart of a method for ADP control by a power station for large-scale energy storage according to an embodiment herein.

DETAILED DESCRIPTION

(3) Embodiments herein will be further elaborated below with reference to the drawings. Note that embodiments below are for illustrating and explaining the present disclosure, and are not intended to limit the present disclosure.

(4) FIG. 1 shows a system for ADP control by a power station for large-scale energy storage according to an embodiment herein. The system may include modules as follows.

(5) The system may include a parameter initialization module. The system for ADP control may include a two-layer structure of an estimation module and a management module, each of which may be built with a three-layer structure of a neural network. A parameter of the management module and the estimation module may include at least one of a discount factor α, network learning rates l.sub.a and l.sub.c, weights W.sub.a and W.sub.c, a maximal number of cycles, an expected error of network training, etc.

(6) An objective control parameter may include at least one of a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of a system for energy storage, a limiting range of a State Of Charge (SOC) of the system for energy storage, a sampling interval Δt, an observation time T, an objective rate of turbulence r.sub.obj.sup.T, a limiting rate of turbulence r.sub.lim.sup.T, etc.

(7) An initial state of a controlled object may include an initial rate of turbulence of wind power, an initial SOC of the system for energy storage, actual wind power output for a present time point, etc. The controlled object may be the power station for large-scale energy storage.

(8) The system may include a data collection and computation module. The actual wind power output as well as charging power and discharging power of the system for energy storage may be acquired. The rate of turbulence of the wind power and the SOC may be computed in real time. It may be determined in real time whether the state parameters are within constraint ranges. The charging power and the discharging power of the system for energy storage may be adjusted when the state parameters are not within the constraint ranges.

(9) The estimation module may be trained as follows. The rate of turbulence of the wind power r.sub.hybrid.sup.T and a control strategy, i.e., a correction ΔP.sub.BESS(t) to power of energy storage may be input to the estimation module. A cost function J.sub.c.sup.WPBESS(t) output by the estimation module may be computed. The estimation module may be trained by building an objective function E.sub.chybrid(t). The weight of the neural network of the estimation module may be updated according to a weight updating formula of the estimation module. The rate of turbulence of the wind power r.sub.hybrid.sup.T and the control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, may have to be normalized to be within [−1, +1] before being sent into the network for computation. The objective function may be built according to the cost function. After the objective function E.sub.chybrid(t) has been built, the estimation module may be retrained by minimizing the objective function, for acquiring, according to the rate of turbulence of the wind power r.sub.hybrid.sup.T collected at a next collecting time point and the control strategy output by the management module, the cost function for next management module training.

(10) The weight W.sub.c of the neural network of the estimation module may be updated by training the estimation module by minimizing the objective function E.sub.chybrid(t) The training may end when the objective function E.sub.chybrid(t) has decreased to a set error or a maximal number of iterations has been reached.

(11) The management module may be trained by minimizing the output J.sub.chybrid(t) of the estimation module. The weight W.sub.a of the neural network of the management module may be updated according to a weight updating formula of the management module. The control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, may be adjusted by training the management module by minimizing the output J.sub.chybrid(t) of the estimation module. The weight W.sub.a of the neural network of the management module may be updated by minimizing the objective function E.sub.ahybrid(t). The training may end when the objective function E.sub.ahybrid(t) has decreased to a set error or a maximal number of iterations has been reached.

(12) Note that the objective function E.sub.chybrid(t) for training the estimation module and the objective function E.sub.ahybrid(t) for training the management module may differ. The E.sub.ahybrid(t) may be positively correlated with a value of the cost function. Accordingly, the management module may be trained by minimizing the E.sub.ahybrid(t).

(13) The system may include an output module. The control strategy for each time point may be stored and output in real time. The smoothing process may be adjusted in real time on line to control the charging power and the discharging power of the system for energy storage. The charging power and the discharging power of the system for energy storage may be controlled by the control strategy.

(14) To sum up, with the present disclosure, both new energy power generation and a system for large-scale energy storage may be incorporated and operated in a grid, and optimal charging and discharging power of the system for energy storage such as a capacity of the system for large-scale energy storage, etc., may be found, by adjusting power of a power station for large-scale energy storage in real time using an ADP algorithm considering both a rate of turbulence of power output of the new energy power generation and the SOC of the system for large-scale energy storage by battery. An ADP algorithm may be independent of an accurate mathematical model of a controlled system or process, and be capable of online self-study to adapt to a change of a system parameter, with great robustness. Therefore, with the present disclosure, the power output of the new energy power generation may be smoothed on line adaptively by adaptive smoothing control with an ADP algorithm, optimizing a result of controlling the system for large-scale energy storage by battery.

(15) FIG. 2 shows a method for ADP control by a power station for large-scale energy storage according to an embodiment herein. The method may include steps as follows.

(16) In step 1, a structure of a system for ADP control, parameters of a management module and an estimation module, and objective control parameter may be set.

(17) The system for ADP control may include a two-layer structure of the estimation module and the management module. Alternatively, the system for ADP control may include a three-layer structure of a model module, the estimation module, and the management module. Each module may be built with a three-layer structure of a neural network. A parameter of the management module and the estimation module may include at least one of a discount factor α, network learning rates l.sub.a and l.sub.c, weights W.sub.a and W.sub.c, a maximal number of cycles, an expected error of network training, etc. An initial weight may be set as a random number within (−1, +1). The learning rates and the discount factor may be selected and adjusted according to an objective result of control. A proper model of a neural network, including at least one of a mode of training, a parameter, a structure, and a type of the network, etc., may be selected via coordinated optimization considering particulars of a controlled object, a convergence speed, an accuracy of a result of computation, etc.

(18) An action network in FIG. 2 may correspond to the neural network of the management module herein. an estimation network in FIG. 2 may be the neural network of the estimation module.

(19) An objective control parameter may include at least one of a capacity of wind power generation P.sub.wp.sup.rated, a capacity W.sub.bat of a system for energy storage, a limiting range of a State Of Charge (SOC) of the system for energy storage, a sampling interval Δt, an observation time T, an objective rate of turbulence r.sub.obj.sup.T, a limiting rate of turbulence r.sub.lim.sup.T, etc.

(20) In step 2, parameter initialization may be performed. An initialized parameter may be imported as an initial state of a controlled object.

(21) The initialized parameter may include at least one of an initial rate of turbulence of wind power, an initial SOC of the system for energy storage, actual wind power output at the present time point, etc.

(22) In step 3, a rate of turbulence of raw wind power r.sub.wp.sup.T for a present time point t may be computed. The raw wind power may be smoothed by controlling a rate of change. A rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed may be computed. Power P.sub.BESS(t) of a system for energy storage may be computed. A State Of Charge (SOC) of the system for energy storage may be computed.

(23) In step 4, initialization may be performed for training the estimation module and the management module.

(24) The rate of turbulence of the raw wind power r.sub.wp.sup.T for the present time point t may be computed and the raw wind power may be smoothed by controlling the rate of change as follows.

(25) The rate of change k(t) of the wind power may be defined according to the raw wind power P.sub.wp(t), a smoothed wind power P.sub.hybrid(t), and the sampling interval Δt according to a formula of

(26) k ( t ) = P wp ( t ) - P hybrid ( t - Δ t ) Δ t . ( 5 )

(27) A rate of turbulence of wind and energy storage hybrid power may be controlled to be within a required range using the control strategy as follows.

(28) For k.sub.hybrid.sup.drop≤k(t)≤k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)  (6).

(29) For k(t)≥k.sub.hybrid.sup.rise,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.rise  (7).

(30) For k(t)<k.sub.hybrid.sup.drop,
P.sub.hybrid(t)=P.sub.wp(t−Δt)+Δt.Math.k.sub.hybrid.sup.drop  (8).

(31) The k.sub.hybrid.sup.rise may be a limiting rate of change that limits rise of wind power output, as defined below. The k.sub.hybrid.sup.drop may be a limiting rate of change that limits drop of the wind power output, as defined below.

(32) 0 k hybrid rise = P wp rated × r obj T T . ( 9 ) k hybrid drop = - P wp rated × r obj T T . ( 10 )

(33) The rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T may be smoothed by controlling the rate of change, using formulae as follows.

(34) r hybrid T = f hybrid ( P hybrid max - P hybrid min P hybrid rated ) . ( 11 ) P hybrid max = max { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } . ( 12 ) P hybrid min = min { P hybrid ( t ) , P hybrid ( t - Δ t ) , .Math. , P hybrid ( t - ( n - 1 ) Δ t ) } . ( 13 ) T = n Δ t . ( 14 )

(35) The P.sub.hybrid.sup.max may be a maximal wind and energy storage hybrid power sampled within the observation time T. The P.sub.hybrid.sup.min may be a minimal wind and energy storage hybrid power sampled within the observation time,

(36) A rate of turbulence of power may be controlled using the change rage to control the rate of turbulence of the wind and energy storage hybrid power smoothed to be less than a specified objective rate within the observation time, as shown by a formula of
r.sub.hybrid.sup.T<r.sub.obj.sup.T  (15),

(37) The r.sub.obj.sup.T may be the objective rate of turbulence of the wind and energy storage hybrid power within the observation time T.

(38) Charging power and discharging power of the system for energy storage may be computed according to the above formulae. The power of energy storage for a time point t may be computed according to a formula of
P.sub.BESS=P.sub.hybrid(t)−P.sub.wp(t)  (16).

(39) The SOC of the energy storage may be computed as follows.

(40) For P.sub.BESS(t)>0, the system for energy storage may be discharged. The SOC may decrease according to a formula of

(41) SOC ( t ) = SOC ( t - Δ t ) - Δ t .Math. P BESS ( t ) .Math. W bat . ( 17 )

(42) For P.sub.BESS(t)<0, the system for energy storage may be charged. The SOC may increase according to a formula of

(43) SOC ( t ) = SOC ( t - Δ t ) + Δ t .Math. P BESS ( t ) .Math. W bat . ( 18 )

(44) The W.sub.bat may be a capacity of the system for energy storage.

(45) In step 5, it may be determined whether the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T meets a constraint of r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T. When it does not meet the constraint, a next step of ADP network training may be performed to seek an optimal correction to the power of energy storage. When it meets the constraint, the system for energy storage may take no action to correct the power of energy storage.

(46) It may be determined whether ADP adjustment or regulation of the power of energy storage is required as follows. Based on the power of energy storage P.sub.BESS(t) of the system for energy storage and the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T acquired by controlling the rate of change, a limiting rate r.sub.lim.sup.T may be newly defined to limit the rate of turbulence of the wind and energy storage hybrid power. The control strategy as follows may be formulated according to the limiting rate and the objective rate of turbulence r.sub.obj.sup.T.

(47) For 0<r.sub.hybrid.sup.T<r.sub.lim.sup.T, the system for energy storage has been outputting too much power. Thus reverse correction as follows may be required.
P.sub.BESS.sup.ADP(t)=P.sub.BESS(t)+ΔP.sub.BESS  (22).

(48) For r.sub.lim.sup.T≤r.sub.hybrid.sup.T<r.sub.obj.sup.T, the system for energy storage has been outputting proper power. Thus no correction may be necessary.
P.sub.BESS.sup.ADP(t)≤P.sub.BESS(t)  (23).

(49) The P.sub.BESS.sup.ADP(t) may be the power of the system for energy storage acquired by ADP of the power of energy storage by controlling the rate of change.

(50) In step 6, a management network may be trained by inputting, to the management network, the state of the controlled object, i.e., the rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.T, the power of energy storage P.sub.BESS(t), etc. The weight of the management network may be updated. The control strategy, i.e., a correction ΔP.sub.BESS(t) to the power of energy storage, may be output.

(51) The management network may be trained as follows.

(52) The management module may be trained by minimizing the output J.sub.chybrid(t) of the estimation module. The weight W.sub.a of the neural network of the management module may be updated according to a weight updating formula of the management module. The control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, may be adjusted by training the management module by minimizing the output J.sub.chybrid(t) of the estimation module. The weight W.sub.a of the neural network of the management module may be updated by minimizing the objective function E.sub.ahybrid(t). The training may end when the objective function E.sub.ahybrid(t) has decreased to a set error or a maximal number of iterations has been reached.

(53) In step 7, an estimation network may be trained by inputting the control strategy and a state of the controlled object to the estimation network. The weight of the estimation network may be updated. A cost function may be output.

(54) The estimation network may be trained as follows.

(55) The rate of turbulence of the wind power r.sub.hybrid.sup.T and a control strategy, i.e., a correction ΔP.sub.BESS(t) to power of energy storage may be input to the estimation module. A cost function J.sub.c.sup.WPBESS(t) output by the estimation module may be computed. The estimation module may be trained by building an objective function E.sub.chybrid(t). The weight of the neural network of the estimation module may be updated according to a weight updating formula of the estimation module. The rate of turbulence of the wind power r.sub.hybrid.sup.T and the control strategy, i.e., the correction ΔP.sub.BESS(t) to the power of energy storage, may have to be normalized to be within [−1, +1] before being sent into the network for computation.

(56) The weight W.sub.c of the neural network of the estimation module may be updated by training the estimation module by minimizing the objective function E.sub.chybrid(t). The training may end when the objective function E.sub.chybrid(t) has decreased to a set error or a maximal number of iterations has been reached.

(57) In step 8, the control strategy for the time may be stored. The state of the controlled object for a next time point t=t+1 may be computed. The steps 5-7 may be repeated.

(58) In step 9, the steps may be cycled until control ends. The rate of turbulence of the wind and energy storage hybrid power r.sub.hybrid.sup.ADP smoothed, the power of energy storage P.sub.BESS.sup.ADP(t), the SOC of the system for energy storage, the control strategy for each time point, etc., may be output.

(59) According to an embodiment herein, a method for ADP control by a power station for large-scale energy storage may include steps as follows.

(60) A rate of turbulence of raw wind power r.sub.wp.sup.T for a present time point t may be computed. The raw wind power may be smoothed by controlling a rate of change. A rate of turbulence of wind and energy storage hybrid power r.sub.hybrid.sup.T smoothed may be computed. Power P.sub.BESS(t) of a system for energy storage may be computed. A State Of Charge (SOC) of the system for energy storage may be computed.

(61) It may be determined whether the r.sub.hybrid.sup.T, the P.sub.BESS(t), and the SOC are within constraint ranges corresponding to an objective control parameter.

(62) When they are not within the constraint ranges, the r.sub.hybrid.sup.T may be input to the management module for the present time point.

(63) The management module for the present time point may output a control strategy according to the r.sub.hybrid.sup.T. The control strategy may be for controlling charging power and discharging power of the system for energy storage of the power station for large-scale energy storage.

(64) The r.sub.hybrid.sup.T and the control strategy for the present time point may be input to the estimation module for the present time point.

(65) The estimation module may output a cost function according to the r.sub.hybrid.sup.T and the control strategy for the present time point.

(66) An objective function for training the estimation module may be built according to the r.sub.hybrid.sup.T and the cost function.

(67) The estimation module may be trained according to the objective function by minimizing the objective function.

(68) The management module may be trained according to the cost function by minimizing the cost function. The estimation module and the management module may be retrained for outputting the control strategy for a next time point.

(69) The P.sub.BESS(t) may include the charging power and the discharging power. It may be determined that the P.sub.BESS(t) is within the constraint range thereof when the charging power is no less than minimal charging power and no greater than maximal charging power allowed, and the discharging power is no less than minimal discharging power and no greater than maximal discharging power allowed. The minimal charging power and/or the minimal discharging power may be 0.

(70) It may be determined that the SOC is within the constraint range thereof when the SOC is no greater than a maximal SOC allowed for the system for energy storage in work, and no less than a minimal SOC allowed for the system for energy storage in work.

(71) It may be determined that the r.sub.hybrid.sup.T is within the constraint range thereof when the r.sub.hybrid.sup.T is less than the objective rate. It may be determined that no adjustment is necessary and no control strategy may have to be output only when all three parameters are within the respective constraint ranges thereof. Otherwise one or more of these parameters may be input to the management module for the present time point to output the control strategy for the present time point for controlling charging power and discharging power of the system for energy storage, and retraining the management module and the estimation module, so as to facilitate subsequent more accurate control of the system for energy storage of the power station for large-scale energy storage.

(72) The control strategy may include a correction to power of energy storage. The system for energy storage of the power station for large-scale energy storage may adjust the charging power and the discharging power per se according to the correction to the power of energy storage.

(73) According to an embodiment herein, a computer-readable storage medium may have stored therein instructions executable by a computer to perform any aforementioned method.

(74) The computer-readable storage medium herein may be various types of storage media. The computer-readable storage medium herein may be a non-transitory storage medium.

(75) As shown in FIG. 1, a system for ADP control by a power station for large-scale energy storage may include at least one of a parameter initialization module, a data collection and computation module, a management module, an estimation module, an output module, etc.

(76) The parameter initialization module may be arranged for: setting an objective control parameter and a structure of a system for ADP control, and sending the objective control parameter to the data collection and computation module.

(77) The data collection and computation module may be arranged for: computing a rate of turbulence of wind and energy storage hybrid power according to the objective control parameter, and sending the rate of turbulence of the wind and energy storage hybrid power to the management module and the estimation module.

(78) The management module may be arranged for: acquiring a correction to power of energy storage according to the rate of turbulence of the wind and energy storage hybrid power, and sending the correction to the power of energy storage to the data collection and computation module, the management module, and the estimation module.

(79) The estimation module may be arranged for: acquiring a cost function according to the rate of turbulence of the wind and energy storage hybrid power, and sending the cost function to the management module.

(80) The output module may be arranged for: outputting a control strategy, a rate of turbulence of wind and energy storage hybrid power smoothed, the power of energy storage, and a State Of Charge (SOC) of a system for energy storage for each time point.

(81) Note that embodiments herein are for illustrating the present disclosure instead of limiting the present disclosure. Any modification made according to the principle of the present disclosure shall be deemed to fall within the scope of the present disclosure.