Smart City Gnosys

Smart city article details

Title	Simulation To Scaled City: Zero-Shot Policy Transfer For Traffic Control Via Autonomous Vehicles
ID_Doc	48908
Authors	Jang K.; Vinitsky E.; Chalaki B.; Remer B.; Beaver L.; Malikopoulos A.A.; Bayen A.
Year	2019
Published	ICCPS 2019 - Proceedings of the 2019 ACM/IEEE International Conference on Cyber-Physical Systems
DOI	http://dx.doi.org/10.1145/3302509.3313784
Abstract	Using deep reinforcement learning, we successfully train a set of two autonomous vehicles to lead a fleet of vehicles onto a roundabout and then transfer this policy from simulation to a scaled city without fine-tuning. We use Flow, a library for deep reinforcement learning in microsimulators, to train two policies, (1) a policy with noise injected into the state and action space and (2) a policy without any injected noise. In simulation, the autonomous vehicles learn an emergent metering behavior for both policies which allows smooth merging. We then directly transfer this policy without any tuning to the University of Delaware’s Scaled Smart City (UDSSC), a 1:25 scale testbed for connected and automated vehicles. We characterize the performance of the transferred policy based on how thoroughly the ramp metering behavior is captured in UDSSC. We show that the noise-free policy results in severe slowdowns and only, occasionally, it exhibits acceptable metering behavior. On the other hand, the noise-injected policy consistently performs an acceptable metering behavior, implying that the noise eventually aids with the zero-shot policy transfer. Finally, the transferred, noise-injected policy leads to a 5% reduction of average travel time and a reduction of 22% in maximum travel time in the UDSSC. Videos of the proposed self-learning controllers can be found at https://sites.google.com/view/iccps-policy-transfer. © 2019 Association for Computing Machinery.
Author Keywords	Autonomous vehicles; Control theory; Cyber-physical systems; Deep learning; Policy Transfer; Reinforcement learning

Similar Articles

Id	Similarity	Authors	Title	Published
62164	0.897	Chalaki B.; Beaver L.E.; Remer B.; Jang K.; Vinitsky E.; Bayen A.M.; Malikopoulos A.A.	Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation To Real-World Via Adversarial Learning	IEEE International Conference on Control and Automation, ICCA, 2020-October (2020)
40923	0.881	Zhang Z.; Zhou B.; Zhang B.; Cheng P.; Lee D.-H.; Hu S.	Optimizing Traffic Signal Control In Mixed Traffic Scenarios: A Predictive Traffic Information-Based Deep Reinforcement Learning Approach	2024 Forum for Innovative Sustainable Transportation Systems, FISTS 2024 (2024)
11516	0.873	Giannini F.; Franze G.; Pupo F.; Fortino G.	Autonomous Vehicles In Smart Cities: A Deep Reinforcement Learning Solution	Proceedings of the 2022 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech 2022 (2022)
26704	0.871	Wu C.; Kreidieh A.R.; Parvate K.; Vinitsky E.; Bayen A.M.	Flow: A Modular Learning Framework For Mixed Autonomy Traffic	IEEE Transactions on Robotics, 38, 2 (2022)
23735	0.87	Sattarzadeh A.R.; Pathirana P.N.	Enhancing Adaptive Traffic Control Systems With Deep Reinforcement Learning And Graphical Models	Proceedings - 2024 IEEE International Conference on Future Machine Learning and Data Science, FMLDS 2024 (2024)
47163	0.869	Guillén A.B.; Barbecho Bautista P.A.; Igartua M.A.	Rutge: Realistic Urban Traffic Generator For Urban Environments Using Deep Reinforcement Learning And Sumo Simulator	International Conference on Vehicle Technology and Intelligent Transport Systems, VEHITS - Proceedings (2025)
1391	0.869	Giannini F.; Franzè G.; Fortino G.; Pupo F.	A Deep Reinforcement Learning Strategy For Intelligent Transportation Systems	Internet of Things, Part F1719 (2024)
33264	0.864	Reda M.; Mountassir F.; Mohamed B.	Introduction To Coordinated Deep Agents For Traffic Signal	2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems, WITS 2019 (2019)
6368	0.863	Dong Y.; Huang H.; Zhang G.; Jin J.	Adaptive Transit Signal Priority Control For Traffic Safety And Efficiency Optimization: A Multi-Objective Deep Reinforcement Learning Framework	Mathematics, 12, 24 (2024)
58613	0.863	Paduraru C.; Paduraru M.; Stefanescu A.	Traffic Light Control Using Reinforcement Learning: A Survey And An Open Source Implementation	International Conference on Vehicle Technology and Intelligent Transport Systems, VEHITS - Proceedings (2022)