Smart City Gnosys

Smart city article details

Title	Robust Reinforcement Learning Strategies With Evolving Curriculum For Efficient Bus Operations In Smart Cities
ID_Doc	46922
Authors	Tang Y.; Qu A.; Jiang X.; Mo B.; Cao S.; Rodriguez J.; Koutsopoulos H.N.; Wu C.; Zhao J.
Year	2024
Published	Smart Cities, 7, 6
DOI	http://dx.doi.org/10.3390/smartcities7060141
Abstract	Highlights: What are the main findings? A novel RL framework is proposed for bus operations, integrating three high-level actions: holding, skipping, and turning around to reduce passenger waiting times. The model incorporates LSTM to capture both Markov and non-Markov processes, enhancing decision-making based on past actions and future predictions. Domain Randomization is used to improve the model’s robustness against unpredictable traffic conditions and passenger demand. Curriculum learning allows the RL agent to efficiently learn complex action spaces, demonstrating improvements over traditional holding-only strategies. What are the implications of the main finding? The proposed RL framework demonstrates that reinforcement learning can effectively handle the complexity of bus operations with curriclumn learning integrated. The integration of domain randomization within RL enhances the model’s adaptability to real-world variations, offering a more robust solution for public transit systems. The combination of holding, skipping, and turning around strategies in the RL model provides greater flexibility and efficiency in bus operations, significantly outperforming traditional single-strategy approaches. This approach highlights the potential of RL to provide scalable and efficient operation in multiagent environments, paving the way for smarter and more adaptive urban transportation systems. Public transit systems are critical to the quality of urban life, and enhancing their efficiency is essential for building cost-effective and sustainable smart cities. Historically, researchers sought reinforcement learning (RL) applications to mitigate bus bunching issues with holding strategies. Nonetheless, these attempts often led to oversimplifications and misalignment with the goal of reducing the total time passengers spent in the system, resulting in less robust or non-optimal solutions. In this study, we introduce a novel setting where each bus, supervised by an RL agent, can appropriately form aggregated policies from three strategies (holding, skipping station, and turning around to serve the opposite direction). It’s difficult to learn them all together, due to learning complexity, we employ domain knowledge and develop a gradually expanding action space curriculum, enabling agents to learn these strategies incrementally. We incorporate Long Short-Term Memory (LSTM) in our model considering the temporal interrelation among these actions. To address the inherent uncertainties of real-world traffic systems, we impose Domain Randomization (DR) on variables such as passenger demand and bus schedules. We conduct extensive numerical experiments with the integration of synthetic and real-world data to evaluate our model. Our methodology proves effective, enhancing bus schedule reliability and reducing total passenger waiting time by over 15%, thereby improving bus operation efficiency and smoothering operations of buses that align with sustainable goals. This work highlights the potential of robust RL combined with curriculum learning for optimizing public transport in smart cities, offering a scalable solution for real-world multi-agent systems. © 2024 by the authors.
Author Keywords	bus oeration; curriculum learning; deep reinforcement learning; intelligent transportation systems; smart cities

Similar Articles

Id	Similarity	Authors	Title	Published
6368	0.876	Dong Y.; Huang H.; Zhang G.; Jin J.	Adaptive Transit Signal Priority Control For Traffic Safety And Efficiency Optimization: A Multi-Objective Deep Reinforcement Learning Framework	Mathematics, 12, 24 (2024)
50634	0.875	Ahmadi K.; Allan V.H.	Smart City: Application Of Multi-Agent Reinforcement Learning Systems In Adaptive Traffic Management	2021 IEEE International Smart Cities Conference, ISC2 2021 (2021)
19978	0.865	Popov S.; Vyhovska I.	Digital Control Tower Model For Public Transport City Network	Lecture Notes in Networks and Systems, 1335 LNNS (2025)
18032	0.865	Singh D.	Deep Reinforcement Learning (Drl) For Real-Time Traffic Management In Smart Cities	2023 International Conference on Communication, Security and Artificial Intelligence, ICCSAI 2023 (2023)
58246	0.863	Abdelmoumene H.; Boussahoul S.	Towards Optimized Dynamic Ridesharing System Through Multi-Objective Reinforcement Learning	2024 IEEE International Multi-Conference on Smart Systems and Green Process, IMC-SSGP 2024 (2024)
25829	0.863	Thamaraiselvi K.; Bohra A.R.; Vishal V.; Sunkara P.S.; Sunku B.; Nityajignesh B.	Exploring Traffic Signal Control: A Comprehensive Survey On Reinforcement Learning Techniques	3rd IEEE International Conference on Industrial Electronics: Developments and Applications, ICIDeA 2025 (2025)
23735	0.861	Sattarzadeh A.R.; Pathirana P.N.	Enhancing Adaptive Traffic Control Systems With Deep Reinforcement Learning And Graphical Models	Proceedings - 2024 IEEE International Conference on Future Machine Learning and Data Science, FMLDS 2024 (2024)
32025	0.859	Fatorachian H.; Kazemi H.	Integrating Learning-Based Solutions In Intelligent Transportation Systems: A Conceptual Framework And Case Studies Validation	Cogent Engineering, 11, 1 (2024)
21429	0.859	Skoropad V.N.; Deđanski S.; Pantović V.; Injac Z.; Vujičić S.; Jovanović-Milenković M.; Jevtić B.; Lukić-Vujadinović V.; Vidojević D.; Bodolo I.	Dynamic Traffic Flow Optimization Using Reinforcement Learning And Predictive Analytics: A Sustainable Approach To Improving Urban Mobility In The City Of Belgrade	Sustainability (Switzerland), 17, 8 (2025)
6325	0.857	Ye Q.; Feng Y.; Macias J.J.E.; Stettler M.; Angeloudis P.	Adaptive Road Configurations For Improved Autonomous Vehicle-Pedestrian Interactions Using Reinforcement Learning	IEEE Transactions on Intelligent Transportation Systems, 24, 2 (2023)