Smart City Gnosys
Smart city article details
| Title | Mvr: Synergizing Large And Vision Transformer For Multimodal Natural Language-Driven Vehicle Retrieval |
|---|---|
| ID_Doc | 38752 |
| Authors | Alzubi T.M.; Mukhtar U.R. |
| Year | 2025 |
| Published | IEEE Access, 13 |
| DOI | http://dx.doi.org/10.1109/ACCESS.2024.3524392 |
| Abstract | In recent years, intelligent transportation systems have played a pivotal role in the development of smart cities, with vehicle retrieval becoming a critical component of traffic management and surveillance. Traditional vehicle retrieval systems rely heavily on image-based matching techniques derived from vehicle re-identification (VReID) tasks. However, these approaches are limited by their dependency on image queries, which may not always be available in real-world scenarios. Natural language (NL)-based vehicle retrieval systems offer a more flexible and accessible alternative by enabling users to query vehicles using textual descriptions. Despite progress in NL-based retrieval, existing methods face challenges in fully capturing multi-granularity information and aligning heterogeneous visual and linguistic inputs. This paper addresses these limitations by proposing a robust Multimodal Vehicle Retrieval (MVR) model that integrates both visual and textual data through a dual-stream architecture. Our model captures complementary local features, alongside global information including motion and environmental context. We utilize InfoNCE and instance losses to align the visual and textual modalities within a shared feature space, while post-processing modules, including Granular Vehicle Feature Refinement and Spatial Relationship Modeling, further enhance retrieval performance by refining vehicle attributes and contextual relationships. Our experiments, conducted on the CityFlow-NL dataset, demonstrate that our model achieves a 35.6% improvement in Mean Reciprocal Rank (MRR), a 41.3% increase in recall at 5 (R@5), and a 22.9% improvement in recall at 10 (R@10) compared to the baseline, and overcomes the inherent challenges of cross-modal retrieval in improving real-world VReID. © 2013 IEEE. |
| Author Keywords | intelligent transportation; multimodal; smart city; Traffic surveillance; vehicle tracking |
Similar Articles
| Id | Similarity | Authors | Title | Published |
|---|---|---|---|---|
12878  | 0.923 | Bo X.; Liu J.; Yang D.; Ma W. | Bridging The Gap: Multi-Granularity Representation Learning For Text-Based Vehicle Retrieval | Complex and Intelligent Systems, 11, 1 (2025) |
| 47389 | 0.916 | Sadiq T.; Omlin C.W. | Scene Retrieval In Traffic Videos With Contrastive Multimodal Learning | Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI (2023) |
| 57381 | 0.914 | Sebastian C.; Imbriaco R.; Meletis P.; Dubbelman G.; Bondarev E.; De With P.H.N. | Tied: A Cycle Consistent Encoder-Decoder Model For Text-To-Image Retrieval | IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2021) |
| 39699 | 0.908 | Du Y.; Zhang B.; Ruan X.; Su F.; Zhao Z.; Chen H. | Omg: Observe Multiple Granularities For Natural Language-Based Vehicle Retrieval | IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2022-June (2022) |
| 7243 | 0.877 | Scribano C.; Sapienza D.; Franchini G.; Verucchi M.; Bertogna M. | All You Can Embed: Natural Language Based Vehicle Retrieval With Spatio-Temporal Transformers | IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2021) |
| 3230 | 0.873 | Shankaranarayan N.; Sowmya Kamath S. | A Novel Approach For Real-Time Vehicle Re-Identification Using Content-Based Image Retrieval With Relevance Feedback | Springer Proceedings in Mathematics and Statistics, 401 (2023) |
| 18090 | 0.873 | Shankaranarayan N.; Kamath Sowmya S. | Deep Vision Based Vehicle Retrieval For Automated Smart Traffic Surveillance Systems | Proceedings - 2022 3rd International Conference on Computation, Automation and Knowledge Management, ICCAKM 2022 (2022) |
| 60942 | 0.859 | Xiang X.; Ma Z.; Li X.; Zhang L.; Zhen X. | Vehicle Re-Identification With Large Separable Kernel Attention And Hybrid Channel Attention | Image and Vision Computing, 155 (2025) |
| 58804 | 0.854 | Taleb H.; Wang C. | Transformer-Based Vehicle Re-Identification With Multiple Details | Proceedings of SPIE - The International Society for Optical Engineering, 13074 (2024) |
| 6641 | 0.854 | Yi X.; Wang Q.; Liu Q.; Rui Y.; Ran B. | Advances In Vehicle Re-Identification Techniques: A Survey | Neurocomputing, 614 (2025) |