Smart City Gnosys
Smart city article details
| Title | Assessment Of Reward Functions For Reinforcement Learning Traffic Signal Control Under Real-World Limitations |
|---|---|
| ID_Doc | 10878 |
| Authors | Egea A.C.; Howell S.; Knutins M.; Connaughton C. |
| Year | 2020 |
| Published | Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, 2020-October |
| DOI | http://dx.doi.org/10.1109/SMC42975.2020.9283498 |
| Abstract | Adaptive traffic signal control is one key avenue for mitigating the growing consequences of traffic congestion. Incumbent solutions such as SCOOT and SCATS require regular and time-consuming calibration, can't optimise well for multiple road use modalities, and require the manual curation of many implementation plans.A recent alternative to these approaches are deep reinforcement learning algorithms, in which an agent learns how to take the most appropriate action for a given state of the system. This is guided by neural networks approximating a reward function that provides feedback to the agent regarding the performance of the actions taken, making it sensitive to the specific reward function chosen. Several authors have surveyed the reward functions used in the literature, but attributing outcome differences to reward function choice across works is problematic as there are many uncontrolled differences, as well as different outcome metrics.This paper compares the performance of agents using different reward functions in a simulation of a junction in Greater Manchester, UK, across various demand profiles, subject to real world constraints: realistic sensor inputs, controllers, calibrated demand, intergreen times and stage sequencing. The reward metrics considered are based on the time spent stopped, lost time, change in lost time, average speed, queue length, junction throughput and variations of these magnitudes. The performance of these reward functions is compared in terms of total waiting time. We find that speed maximisation resulted in the lowest average waiting times across all demand levels, displaying significantly better performance than other rewards previously introduced in the literature. © 2020 IEEE. |
| Author Keywords | Agent-Based Modeling; Reinforcement Learning; Smart Cities; Urban Traffic Control |
Similar Articles
| Id | Similarity | Authors | Title | Published |
|---|---|---|---|---|
25605  | 0.916 | Paul A.; Mitra S. | Exploring Reward Efficacy In Traffic Management Using Deep Reinforcement Learning In Intelligent Transportation System | ETRI Journal, 44, 2 (2022) |
| 40923 | 0.913 | Zhang Z.; Zhou B.; Zhang B.; Cheng P.; Lee D.-H.; Hu S. | Optimizing Traffic Signal Control In Mixed Traffic Scenarios: A Predictive Traffic Information-Based Deep Reinforcement Learning Approach | 2024 Forum for Innovative Sustainable Transportation Systems, FISTS 2024 (2024) |
| 23735 | 0.901 | Sattarzadeh A.R.; Pathirana P.N. | Enhancing Adaptive Traffic Control Systems With Deep Reinforcement Learning And Graphical Models | Proceedings - 2024 IEEE International Conference on Future Machine Learning and Data Science, FMLDS 2024 (2024) |
| 25829 | 0.891 | Thamaraiselvi K.; Bohra A.R.; Vishal V.; Sunkara P.S.; Sunku B.; Nityajignesh B. | Exploring Traffic Signal Control: A Comprehensive Survey On Reinforcement Learning Techniques | 3rd IEEE International Conference on Industrial Electronics: Developments and Applications, ICIDeA 2025 (2025) |
| 58613 | 0.891 | Paduraru C.; Paduraru M.; Stefanescu A. | Traffic Light Control Using Reinforcement Learning: A Survey And An Open Source Implementation | International Conference on Vehicle Technology and Intelligent Transport Systems, VEHITS - Proceedings (2022) |
| 44895 | 0.885 | Barta Z.; Kovács S.; Botzheim J. | Reinforcement Learning-Based Cooperative Traffic Control System | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 14811 LNAI (2024) |
| 33264 | 0.878 | Reda M.; Mountassir F.; Mohamed B. | Introduction To Coordinated Deep Agents For Traffic Signal | 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems, WITS 2019 (2019) |
| 6368 | 0.875 | Dong Y.; Huang H.; Zhang G.; Jin J. | Adaptive Transit Signal Priority Control For Traffic Safety And Efficiency Optimization: A Multi-Objective Deep Reinforcement Learning Framework | Mathematics, 12, 24 (2024) |
| 12771 | 0.874 | Cao M.; Li V.O.K.; Shuai Q. | Book Your Green Wave: Exploiting Navigation Information For Intelligent Traffic Signal Control | IEEE Transactions on Vehicular Technology, 71, 8 (2022) |
| 57625 | 0.874 | Ait Ouallane A.; Bahnasse A.; Bakali A.; Talea M. | Toward A Smart City: Reinforcement Learning For Traffic Light Control | Lecture Notes in Networks and Systems, 629 LNNS (2023) |