原文信息
A coordinated active and reactive power optimization approach for multi-microgrids connected to distribution networks with multi-actor-attention-critic deep reinforcement learning
原文链接：
https://www.sciencedirect.com/science/article/pii/S0306261924012534
Highlights
(1) 构建了一种面向多微网群运行优化的强化学习优化模型，构建了离散动作变量和连续动作变量两种多智能体，通过多智能体协同使得连续和离散的有功/无功设备能够参与多微网群优化。
(2) 提出引入注意力机制的多智能体强化学习算法（Multi-Actor-Attention-Critic，MAAC）以解决高维输入下智能体因维数爆炸导致的收敛性问题。
(3) 将迁移学习嵌入到多智能体深度强化训练过程以改善多智能体模型在电网运行工况变化场景下的训练性能。
Research gap
针对高维、非线性、变工况条件下含多微网的配电网分布式优化模型的精确、高效求解难题，提出一种基于MAAC深度强化学习的含多微网的配电网有功无功协同优化方法，显著改善了大规模非凸模型分布式优化求解的可行性和计算效率。
Abstract
As a promising approach to managing distributed energy, the use of microgrids has attracted significant attention among those managing continuous connections to distribution networks. However, the barriers of the data sharing among different microgrids, the uncertainty of the distributed renewable sources and loads, and the nonlinear optimization of power flow make traditional model-based optimization methods difficult to be applied. In this paper, a data-driven coordinated active and reactive power optimization method is proposed for distribution networks with multi-microgrids. A multi-agent deep reinforcement learning (MADRL) method is used to protect the data privacy of each microgrids. Moreover, attention mechanism, which pays attention to crucial information, is presented to overcome the problem of slow convergence caused by the dimensionality explosion of the optimized variables. Two types of agents, controlling discrete action and continuous action devices, respectively, are formulated in coordinated optimization, which reduces voltage violations and improves the system operation efficiency. In addition, in order to improve the performance of the online agent model under variable operation conditions, the transfer learning is embedded in the training process of the MADRL. The proposed method is verified on a modified IEEE 33-bus distribution network with nine microgrids.
Keywords
Distribution networks with multiple microgrids
Coordinated active and reactive power optimization
Attention mechanisms
Multi-agent deep reinforcement learning
Transfer learning
Graphics
图1 基于多智能体深度强化学习的有功-无功协调优化架构
图4 MAAC框架
图8 多智能体算法收敛曲线
图15 电功率平衡情况
图19 微网与配网交互功率
作者简介
作者简介：
董雷，华北电力大学教授，博士生导师，主要从事电网优化调度与运行控制、综合能源系统优化、人工智能在电力系统中的应用等领域的研究。作为负责人承担国家自然科学基金面上项目 1项，国家重点研发计划子课题2项，担任中国电机工程学会人工智能专委会委员。曾荣获北京市科技进步一等奖，中国电力科学技术进步一等奖、北京市教学成果二等奖。
林灏，华北电力大学硕士研究生，主要从事配电网优化运行与控制方面的研究工作。
关于Applied Energy
本期小编：任郡枝；审核人：董雷
《Applied Energy》是世界能源领域著名学术期刊，在全球出版巨头爱思唯尔 (Elsevier) 旗下，1975年创刊，影响因子10.1，CiteScore 21.2，本刊旨在为清洁能源转换技术、能源过程和系统优化、能源效率、智慧能源、环境污染物及温室气体减排、能源与其他学科交叉融合、以及能源可持续发展等领域提供交流分享和合作的平台。开源（Open Access）姊妹新刊《Advances in Applied Energy》影响因子13.0，CiteScore 23.9。全部论文可以免费下载。在《Applied Energy》的成功经验基础上，致力于发表应用能源领域顶尖科研成果，并为广大科研人员提供一个快速权威的学术交流和发表平台，欢迎关注！
公众号团队小编招募长期开放，欢迎发送自我简介（含教育背景、研究方向等内容）至wechat@applied-energy.org
点击“阅读原文”
喜欢我们的内容？
点个“赞”或者“再看”支持下吧！
阅读原文
