Smart City Gnosys
Smart city article details
| Title | Marlisa: Multi-Agent Reinforcement Learning With Iterative Sequential Action Selection For Load Shaping Of Grid-Interactive Connected Buildings |
|---|---|
| ID_Doc | 36438 |
| Authors | Vazquez-Canteli J.R.; Henze G.; Nagy Z. |
| Year | 2020 |
| Published | BuildSys 2020 - Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation |
| DOI | http://dx.doi.org/10.1145/3408308.3427604 |
| Abstract | We demonstrate that multi-agent reinforcement learning (RL) controllers can cooperate to provide more effective load shaping in a model-free, decentralized, and scalable way with very limited sharing of anonymous information. Rapid urbanization, increasing electrification, the integration of renewable energy resources, and the potential shift towards electric vehicles create new challenges for the planning and control of energy systems in smart cities. Energy storage resources can help better align peaks of renewable energy generation with peaks of electricity consumption and flatten the curve of electricity demand. Model-based controllers, such as MPC, require developing models of the systems controlled, which is often not cost-effective or scalable. Model-free controllers, such as RL, have the potential to provide good control policies cost-effectively and leverage the use of historical data for training. However, it is unclear how RL algorithms can control a multitude of energy systems in a scalable coordinated way. In this paper, we introduce MARLISA, a controller that combines multi-agent RL with our proposed iterative sequential action selection algorithm for load shaping in urban energy systems. This approach uses a reward function with individual and collective goals, and the agents predict their own future electricity consumption and share this information with each other following a leader-follower schema. The RL agents are tested in four groups of nine simulated buildings, with each group located in a different climate. The buildings have diverse load and domestic hot water profiles, PV panels, thermal storage devices, heat pumps, and electric heaters. The agents are evaluated on the average of five normalized metrics: annual net electric consumption, 1 - load factor, average daily peak demand, annual peak demand, and ramping. MARLISA achieves superior results over multiple independent/uncooperative RL agents using the same reward function. Our results outperformed a manually optimized rule-based controller (RBC) benchmark by reducing the average daily peak load by 15%, ramping by 35%, and increasing the load factor by 10%. A multi-year case study on real weather data shows that MARLISA significantly outperforms the RBC in within a year and converges in less than 2 years. Combining MARLISA and the RBC for the first year improves overall initial performance by learning from the RBC rather than random exploration. © 2020 ACM. |
| Author Keywords | demand response; microgrid; multi-agent coordination; Reinforcement learning |
Similar Articles
| Id | Similarity | Authors | Title | Published |
|---|---|---|---|---|
38102  | 0.906 | Vazquez-Canteli J.; Detjeen T.; Henze G.; Kämpf J.; Nagy Z. | Multi-Agent Reinforcement Learning For Adaptive Demand Response In Smart Cities | Journal of Physics: Conference Series, 1343, 1 (2019) |
| 11474 | 0.904 | Özkan E.; Kök I.; Özdemir S. | Autonomous Micro-Grids: A Reinforcement Learning-Based Energy Management Model In Smart Cities | 2023 International Symposium on Networks, Computers and Communications, ISNCC 2023 (2023) |
| 7714 | 0.901 | Ludolfinger U.; Martens M. | An Autonomous Energy Management Concept For Sustainable Smart Cities | 2023 IEEE European Technology and Engineering Management Summit, E-TEMS 2023 - Conference Proceedings (2023) |
| 38091 | 0.897 | Tariq S.; Ali U.; Kim S.; Yoo C. | Multi-Agent Distributed Reinforcement Learning For Energy-Efficient Thermal Comfort Control In Multi-Zone Buildings With Diverse Occupancy Patterns | Energy, 332 (2025) |
| 54163 | 0.894 | Tungom C.E.; Wang H.; Beata K.; Niu B. | Swoam: Swarm Optimized Agents For Energy Management In Grid-Interactive Connected Buildings | Energy, 301 (2024) |
| 1412 | 0.882 | Tomin N.; Kolosok I.; Kurbatsky V.; Korlina E. | A Demand-Response Approach For Hvac Systems Using Internet Of Energy Concept | Lecture Notes in Networks and Systems, 846 (2024) |
| 26378 | 0.876 | Khanna A.; Maheshwari P. | Federated Multi-Agent Reinforcement Learning For Incentive-Based Drs Over Blockchain Enabled Microgrids | 2024 7th International Conference on Signal Processing and Information Security, ICSPIS 2024 (2024) |
| 54129 | 0.873 | Tungom C.E.; Niu B.; Wang H. | Swapp: Swarm Precision Policy Optimization With Dynamic Action Bound Adjustment For Energy Management In Smart Cities | Applied Energy, 377 (2025) |
| 44356 | 0.869 | Faghri S.; Tahami H.; Amini R.; Katiraee H.; Godazi Langeroudi A.S.; Alinejad M.; Ghasempour Nejati M. | Real-Time Energy Flexibility Optimization Of Grid-Connected Smart Building Communities With Deep Reinforcement Learning | Sustainable Cities and Society, 119 (2025) |
| 20702 | 0.868 | Kumar P.L.; Jayanthi M.; Singh J.; Gupta M.; Bobba P.B.; Albawi A.; Shubhra | Distributed Reinforcement Learning Framework For Collaborative Energy Management In Connected Hybrid Electric Vehicle Ecosystems | 2025 International Conference on Intelligent Control, Computing and Communications, IC3 2025 (2025) |