| Abstract |
With the improvement of real-time requirements for big data analysis of the Internet of Things, the cloud big data analysis method controlled by the center cannot meet the real-time and accuracy requirements due to its large response delay, high cost, and low prediction accuracy in specific environments. This paper proposes an edge-side big data analysis and predictive modeling method under massive real-time data such as sensor data, streaming data and other scenarios. This method trains small data samples on the edge side, multi-accesses the edge side for distributed distribution according to specific application scenarios and conducts model learning, models training and inference predictive analysis. Firstly, by combining big data analysis and edge computing, a theoretical paradigm framework for big data analysis and prediction modeling on the edge side and cloud collaboration is proposed. Secondly, the edge side big data analysis and prediction training algorithm and tuning mechanism are designed. Finally, the prototype of the training and evaluation system for edge-side big data analysis is realized. Experimental results in a test environment with hundreds of nodes show that in real-time big data scenarios, compared with cloud training, the performance and efficiency of the edge-side big data training proposed in this paper is increased by an average of 3. 95 times, and the network traffic is reduced by 88. 7% . The prediction accuracy, recall rate and F1 value of the collaborative training model can be improved by 3% - 9% compared with the traditional training method, and the response delay of request prediction is reduced by 67. 5% . The method in this paper can be effectively applied to scientific computing, smart finance, autonomous driving, security monitoring, data security, smart factories, smart cities and other fields. © 2022 Inst. of Scientific and Technical Information of China. All rights reserved. |
