摘要
本文提出了一种新型的集成门控循环单元（GRU）和深度强化学习（DRL）的住宅虚拟电厂（RVPP）调度方法。在所提方法中，GRU集成的DRL算法引导RVPP有效参与日前和实时市场，降低终端用户的购电成本和消费风险。为了避免在训练过程中违反约束条件，引入拉格朗日松弛技术，将约束马尔可夫决策过程（CMDP）转化为无约束优化问题，从而无需整定惩罚系数并保证严格满足约束条件。此外，为了增强基于约束柔性动作-评价器 （CSAC） 的 RVPP 调度方法的可扩展性，设计了一种完全分布式调度架构，以实现 RDER 的即插即用。所构建RVPP场景中的算例分析验证了所提出的方法在提高RDER对电价的响应能力、平衡电网供需和确保用户舒适度方面的有效性。
GRU-integrated constrained soft actor-critic learning enabled fully distributed scheduling strategy for residential virtual power plant
集成GRU的约束柔性动作-评价器住宅虚拟电厂全分布式调度策略
Hui Liu,Yundan Cheng,Yanhui Xu,Guanqun Sun, Rusi Chen,Xiaodong Yu
1.College of Electrical Engineering,Sichuan University,Chengdu 610065,P.R.China
2.State Grid Sichuan Comprehensive Energy Service Co.Ltd.,Chengdu 610072,P.R.China
收听作者1分钟语音介绍
Abstract
Keywords
Residential virtual power plant; Residential distributed energy resource; Constrained soft actor-critic; Fully distributed scheduling strategy
Fig. 1 Decentralized RVPP model
Fig. 2 GRU-CSAC-based RVPP scheduling structure
Fig. 3 The process of 10,000 steps of training for different appliances, where (a)-(c) represent the rewards, costs, and Lagrangian coefficients of energy storage system (ESS); (d)-(f) represent the rewards, comfort costs, and Lagrangian coefficients of electric vehicle (EV); (g)-(i) represent the rewards, comfort costs, and λC of electric water heater (EWH); (j)-(l) represent the rewards, comfort costs, and Lagrangian coefficients of air conditioner (AC)
Fig. 4 Rewards and single-step iteration time for CSAC, SAC, TD3, and DDPG algorithms
Fig. 5 Cost for CSAC, SAC, TD3, and DDPG algorithms
Fig. 6 Loss and training time of GRU and LSTM
Fig. 7 The trading power in DAM and RBM of the RVPP and price curves for RBM
Fig. 8 Power, SOC, and action strategy of ESS
Fig. 9 Power, SOC, and action strategy of EV
Fig. 10 Power, water temperature, and action strategy of the EWH
Fig. 11 Power, reduction factor, and action strategy of the air conditioner
本文引文信息
Deng X Y, Chen Y D, Fan D C, et al. (2023) GRU-integrated constrained soft actor-critic learning enabled fully distributed scheduling strategy for residential virtual power plant, Global Energy Interconnection, 7(2): 117-129
邓孝云，陈永东，范东川，等 (2023) 集成GRU的约束柔性动作-评价器住宅虚拟电厂全分布式调度策略. 全球能源互联网（英文）, 7(2): 117-129
Biographies
Xiaoyun Deng
Xiaoyun Deng received bachelor’s degree at Sichuan University,Chengdu,2021.He is working towards master’s Sichuan University,Chengdu.His research interests includes deep reinforcement learning,virtual power plant and edge intelligence.
Yongdong Chen
Yongdong Chen received the B.S.degree in electrical engineering and automation from Chongqing Jiaotong University,Chongqing,China,in 2018,and the M.S.degree in electrical engineering from Xiangtan University,Xiangtan,Ching,in 2021.He is currently pursuing the Ph.D.degree in electrical engineering with Sichuan University,Chengdu,China.His research interests include operational control of active distribution networks and distributed resources optimization driven by edge intelligence.
Dongchuan Fan
Dongchuan Fan is currently pursuing the master’s degree in electrical engineering with Sichuan University.The main research directions are demand side management and artificial intelligence and their applications in power systems.
Youbo Liu
Youbo Liu graduated from Sichuan University,currently serves as a professor and doctoral supervisor at Sichuan University,and is a reserve for academic and technical leaders in Sichuan Province.He is mainly engaged in research in the fields of artificial intelligence in power systems,low-carbon electricity markets,distributed energy and energy storage,and active distribution networks.
Chao Ma
Chao Ma received PhD degree at Sichuan University,Chengdu,2013.(He received master degree at Sichuan University,Chengdu,2009;bachelor degree at Sichuan University,Chengdu,2006).He is working in State Grid Sichuan Comprehensive Energy Service Co.Ltd,Chengdu.His research interests includes Now power system and Energy storage.
编辑：王彦博
审核：王 伟
郑重声明
根据国家版权局相关规定，纸媒、网站、微博、微信公众号转载、摘编本微信作品，需包含本微信名称、二维码等关键信息，并在文首注明《全球能源互联网》原创。
个人请按本微信原文转发、分享。
分享优秀论文，帮小编点个 “在看” 哦
