Computer and Modernization ›› 2025, Vol. 0 ›› Issue (01): 15-19.doi: 10.3969/j.issn.1006-2475.2025.01.003
Previous Articles Next Articles
Online:
Published:
Abstract: With the popularization and application of IoT technology, a large amount of real-time data needs to be processed and analyzed. Due to the massive and real-time characteristics of IoT data, traditional databases cannot meet the requirements of data storage scale and data processing efficiency, this article proposes a distributed real-time IoT data processing system based on StarRocks. The system utilizes the distributed architecture of StarRocks to build underlying data storage. By introducing message queues and data merging batch submission technology, it ensures fast data writing. At the same time, through storage strategies optimization, index optimization, and materialized view technology, rapid processing and querying of large-scale real-time data have been achieved. The powerful data compression capability of the system also effectively saves data storage space. This framework supports horizontal scaling in data storage scale, improving availability and robustness. Through experimental analysis, the system has significant advantages over traditional distributed databases in terms of data writing, data querying, and data compression.
Key words: StarRocks, real time data processing, distributed system, data compression, query optimization
CLC Number:
TP311 
DONG Yizhou1, PAN Weihua2, ZHANG Nan1, MENG Zhuang1. Real Time IoT Data Processing System Based on StarRocks[J]. Computer and Modernization, 2025, 0(01): 15-19.
0 / / Recommend
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.c-a-m.org.cn/EN/10.3969/j.issn.1006-2475.2025.01.003
http://www.c-a-m.org.cn/EN/Y2025/V0/I01/15