Smart City Gnosys

Smart city article details

Title	Edgeci: Distributed Workload Assignment And Model Partitioning For Cnn Inference On Edge Clusters
ID_Doc	21857
Authors	Chen Y.; Luo T.; Fang W.; Xiong N.N.
Year	2024
Published	ACM Transactions on Internet Technology, 24, 2
DOI	http://dx.doi.org/10.1145/3656041
Abstract	Deep learning technology has grown significantly in new application scenarios such as smart cities and driver-less vehicles, but its deployment needs to consume a lot of resources. It is usually difficult to execute inference task solely on resource-constrained Intelligent Internet-of-Things (IoT) devices to meet strictly service delay requirements. CNN-based inference task is usually offloaded to the edge server or cloud. However, it may lead to unstable performance and privacy leaks. To address the above challenges, this article aims to design a low latency distributed inference framework, EdgeCI, which assigns inference tasks to locally idle, connected, and resource-constrained IoT device cluster networks. EdgeCI exploits two key optimization knobs, including: (1) Auction-based Workload Assignment Scheme (AWAS), which achieves the workload balance by assigning each workload partition to the more matching IoT device; (2) Fused-Layer parallelization strategy based on non-recursive Dynamic Programming (DPFL), which is aimed at further minimizing the inference time. We have implemented EdgeCI based on PyTorch and evaluated its performance with VGG-16 and ResNet-34 image recognition models. The experimental results prove that our proposed AWAS and DPFL outperform the typical state-of-the-art solutions. When they are well combined, EdgeCI can improve inference speed by 34.72% to 43.52%. EdgeCI outperforms the state-of-the art approaches on our edge cluster. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Author Keywords	auction algorithm; Edge computing; fused-layer parallelization; high-speed networks; inference acceleration; workload assignment

Similar Articles

Id	Similarity	Authors	Title	Published
20553	0.904	Li Q.; Huang L.; Tong Z.; Du T.-T.; Zhang J.; Wang S.-C.	Dissec: A Distributed Deep Neural Network Inference Scheduling Strategy For Edge Clusters	Neurocomputing, 500 (2022)
6307	0.885	Zhou L.; Samavatian M.H.; Bacha A.; Majumdar S.; Teodorescu R.	Adaptive Parallel Execution Of Deep Neural Networks On Heterogeneous Edge Devices	Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, SEC 2019 (2019)
21859	0.881	Xue F.; Fang W.; Xu W.; Wang Q.; Ma X.; Ding Y.	Edgeld: Locally Distributed Deep Learning Inference On Edge Device Clusters	Proceedings - 2020 IEEE 22nd International Conference on High Performance Computing and Communications, IEEE 18th International Conference on Smart City and IEEE 6th International Conference on Data Science and Systems, HPCC-SmartCity-DSS 2020 (2020)
41334	0.873	Xiao D.; Wang X.; Yang Z.; Huang C.	Partial Distributed Deep Learning Inference Model For Image Based Edge Device Cluster	Proceedings of 2024 8th International Conference on Electronic Information Technology and Computer Engineering, EITCE 2024 (2025)
5251	0.869	Hoffpauir K.; Simmons J.; Schmidt N.; Pittala R.; Briggs I.; Makani S.; Jararweh Y.	A Survey On Edge Intelligence And Lightweight Machine Learning Support For Future Applications And Services	Journal of Data and Information Quality, 15, 2 (2023)
21748	0.868	Yang Y.-T.; Wei H.-Y.	Edge Computing And Networking Resource Management For Decomposable Deep Learning: An Auction-Based Approach	2021 22nd Asia-Pacific Network Operations and Management Symposium, APNOMS 2021 (2021)
32524	0.868	Vigenesh M.; Katyal A.; Hemalatha S.; Ahluwalia G.; Kukreja M.; Mathurkar P.	Intelligent Resource Scheduling For Edge-Integrated Iot Using Deep Learning	2024 IEEE 4th International Conference on ICT in Business Industry and Government, ICTBIG 2024 (2024)
5117	0.866	Bellavista P.; Chatzimisios P.; Foschini L.; Paradisioti M.; Scotece D.	A Support Infrastructure For Machine Learning At The Edge In Smart City Surveillance	Proceedings - IEEE Symposium on Computers and Communications, 2019-June (2019)
29005	0.858	Ilhan F.; Tekin S.F.; Hu S.; Huang T.; Chow K.-H.; Liu L.	Hierarchical Deep Neural Network Inference For Device-Edge-Cloud Systems	ACM Web Conference 2023 - Companion of the World Wide Web Conference, WWW 2023 (2023)
7415	0.853	Moghaddasi K.; Rajabi S.; Gharehchopogh F.S.; Ghaffari A.	An Advanced Deep Reinforcement Learning Algorithm For Three-Layer D2D-Edge-Cloud Computing Architecture For Efficient Task Offloading In The Internet Of Things	Sustainable Computing: Informatics and Systems, 43 (2024)