原文信息
Voltage control of distribution grid with district cooling systems based on scenario-classified reinforcement learning
原文链接：
https://www.sciencedirect.com/science/article/pii/S0306261924017987
Highlights
(1) The active power of district cooling systems and the reactive power of PV inverters are cooperatively controlled for voltage regulations by RL method.
(2) A compensator-based RL scheme is developed to address the reward-sparse issue for enhancing the RL training robustness.
(3)Ascenario-classified experience replay is introduced to the sampling process to handle the biased distribution issue.
Research gap
Abstract
Modern distribution grids are currently being challenged by frequent and sizable voltage fluctuations, due mainly to the increasing deployment of renewable generation. Considering equipment characteristics, traditional devices (e.g., on-load tap changers) to maintain bus voltages cannot provide frequent regulations. To deal with short-term voltage fluctuations, this paper proposes to cooperatively control reactive and active power through PV inverters and district cooling systems. However, traditional voltage control optimization relies heavily on accurate physical models (e.g., network topology), and brings huge computation burdens with enlarging system scale. In this context, this paper adopts model-free reinforcement learning (RL) to solve the controller without prior knowledge of system models. However, because voltage violations occur not frequently in practice, the irregular occurrence brings sparse reward and biased-distribution experience issues in RL training. Hence, on top of the traditional actor–critic structure, we propose two improvements: 1) a compensator module is designed to cope with the sparse reward issue; 2) a scenario-classified experience replay method is proposed for RL training sampling, which can correct the experience distribution to improve training efficiency for a typical scenario with violated voltages. Numerical studies on a 33-bus network show that, the proposed method can smooth voltage fluctuations better, with negligible temperature impacts on demand-side users.
Keywords
Voltage control
Distributed network
Transient stability
Deep reinforcement learning
Renewable energy
District cooling system
Power quality improvement
Graphics
Fig. 1. Concept graphic of DCS thermal dynamics during the voltage control process.
Fig. 2. The algorithm framework of the proposed SCER-TD3. The dashed line indicates data exchange between modules while the solid line represents module internal workflow.
Fig. 3. IEEE-33 bus system with DCSs and PVs.
Fig. 4. Compensator training losses.
Fig. 5. Training episode rewards of different RL agents.
Fig. 6. Boxlots of voltage control results by different RL methods: (a) Dail
Fig. 7.
Voltage control results of an example day using different RL methods: (a) TD3, (b) CTD3, (c) PER-CTD3, and (d) SCER-CTD3.
Fig. 8. Indoor temperature deviations of all the buildings based on different scenarios:
(a) DCS provides voltage regulation services based on SCER-CTD3; (b) DCS does not
provide voltage regulation services.
Daily max voltage violation.
团队介绍
澳门大学智慧城市物联网国家重点实验室于2018年成立，是我国第一个智慧城市、物联网领域的国家重点实验室，是澳门大学第三间国家重点实验室。实验室围绕五个方向下设子研究室，包括：智能传感器与网络通信、城市大数据与智能技术、智慧能源、智能交通、城市公共安全与灾害防治。
【第一作者】余佩佩，上海电力大学讲师，分别于2016年和2019年获浙江大学数学系本硕学位，2024年获澳门大学博士学位。目前致力于研究安全强化学习、电网需求响应、区域供冷系统及人工智能控制等相关领域。并基于相关研究成果，于2021年获南网AI电力调度大赛唯一创新奖及一等奖。在IEEE Trans. Smart Grid、IEEE Trans. Power Systems等国际顶级期刊发表一作论文5篇，获2021年iSPEC会议最佳论文奖。
【通讯作者】张洪财，澳门大学智慧城市物联网国家重点实验室助理教授、博士生导师。2013年、2018年分别获得清华大学电气工程学士学位、博士学位，曾任美国加州大学伯克利分校博士后研究员，美国劳伦斯伯克利国家实验室访问研究员。主要从事城市综合能源系统、能源物联网、电气化交通等相关领域研究工作。主持国家自然科学基金及澳门科技发展基金等国家及省部级项目4项。以主要作者身份在Nature Energy、IEEE Transactions on Smart Grid等国际SCI索引期刊发表学术论文30余篇，其中ESI高被引论文2篇、热点论文1篇。曾获澳门自然科学二等奖、斯坦福大学全球前2%顶尖科学家（年度影响力）、国际会议EI2-2022最佳论文奖、国际会议ISPEC-2021最佳论文奖、国际会议EVS34-2021优秀论文奖等。担任《Journal of Modern Power Systems and Clean Energy》副编辑、《IET Electrical Systems in Transportation》副编辑、《iEnergy》副编辑、《中国电力》青年编委、中国电工技术学会青年工作委员会委员、IEEE PES (中国) 电动汽车与能源交通系统融合技术分委会秘书长、IEEE能源互联网协调委员会委员等。
宋永华教授担任澳门大学智慧城市物联网国家重点实验室主任、智慧能源研究室学术带头人。宋永华教授长期致力于电力系统低碳、安全与优化运行研究，在新能源电力系统需求侧负荷调控、复杂电网安全分析与控制、电力系统经济优化运行方面取得了系统性的理论与应用创新成果。于2004年、2008年、2019年分别当选英国皇家工程院院士、国际电气与电子工程师学会会士（IEEE Fellow）、欧洲科学院外籍院士。获国家科技进步二等奖1项、省部级一等奖4项，及何梁何利基金科学与技术进步奖、光华工程科技奖和全国创先争优奖。
关于Applied Energy
本期小编：余佩佩；审核人：李辉
《Applied Energy》是世界能源领域著名学术期刊，在全球出版巨头爱思唯尔 (Elsevier) 旗下，1975年创刊，影响因子10.1，CiteScore 21.2，本刊旨在为清洁能源转换技术、能源过程和系统优化、能源效率、智慧能源、环境污染物及温室气体减排、能源与其他学科交叉融合、以及能源可持续发展等领域提供交流分享和合作的平台。开源（Open Access）姊妹新刊《Advances in Applied Energy》影响因子13.0，CiteScore 23.9。全部论文可以免费下载。在《Applied Energy》的成功经验基础上，致力于发表应用能源领域顶尖科研成果，并为广大科研人员提供一个快速权威的学术交流和发表平台，欢迎关注！
公众号团队小编招募长期开放，欢迎发送自我简介（含教育背景、研究方向等内容）至wechat@applied-energy.org
点击“阅读原文”
喜欢我们的内容？
点个“赞”或者“再看”支持下吧！
阅读原文
