Data Compression Algorithm in Real-time Database

doi:10.3969/j.issn.1006-2475.2016.06.025

Abstract

Abstract: To handle the problem of storage caused by large amount of data in real-time database, a classification data compression algorithm is proposed, which can achieve the lossless and efficient compression of real-time database data. First, the data of the real-time database are divided into three parts: time stamp, quality code and value. Then the LZ78 and LZW data compression algorithms are fused according to the characteristics of each data. Experimental results show that this algorithm can improve the actual storage capacity of database and the real-time performance of real-time database.


Key words: real-time database, data compression algorithm, LZ78 algorithm, LZW algorithm

CLC Number:

TP311.13

DAI Yang, CHEN Fang. Data Compression Algorithm in Real-time Database[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2016.06.025.

References

［1］潘玉伟. 基于实时数据库系统的IT资产性能监控解决方案［J］. 中国新通信, 2015(16):113-115.
［2］张少敏,李志雄,王保义,等. 一种面向智能电网的实时数据库数据完整性恢复方法［J］. 电力系统自动化, 2013,37(13):93-98.
［3］于松涛,王晓琨,赵利强,等. 基于容差动态调整的旋转门(SDT)改进算法［J］. 北京化工大学学报(自然科学版), 2013,40(3):109-113.
［4］ Chang Lily, He Xudong. A model transformation approach for verifying multi-agent systems using SPIN［C］// Proceedings of the 2011 ACM Symposium on Applied Computing. 2011:37-42.
［5］ Cai Jin-yi, Kowalczyk M, Williams T. Gadgets and anti-gadgets leading to a complexity dichotomy［C］// Proceedings of the 3rd Innovations in Theoretical Computer Science Conference. 2012:452-467.
［6］ Belcour L, Soler C. Frequency based kernel estimation for progressive photon mapping［C］// Proceedings of SIGGRAPH Asia 2011 Posters. 2011: Article No. 47.
［7］蒋华,李明珍,王鑫,等. 一种基于概率包标记的PPM算法改进方案［J］. 山东大学学报(理学版), 2011,46(9):85-88.
［8］ Rottenstreich O, Tapolcai J. Lossy compression of packet classifiers［C］// Proceedings of the 11th ACM/IEEE Symposium on Architectures for Networking and Communications Systems. 2015:39-50.
［9］ Navarro G. Implementing the LZ-index: Theory versus practice［J］. Journal of Experimental Algorithmics, 2009,13(2):Article No. 2.
［10］Bannai H, Inenaga S, Takeda M, et al. Efficient LZ78 factorization of grammar compressed text string［C］// Proceedings of the 19th International Conference on String Processing and Information Retrieval. 2012:86-98.
［11］邹学玉,冯振,张少华,等. 基于LZW算法的声波测井数据压缩研究［J］. 测井技术, 2013,37(3):294-296.
［12］Li Haifeng, Zhang Tian, Ma Lin, et al. Digital media information compression through auditory content analysis［C］// Proceedings of the 2nd International Conference on Internet Multimedia Computing and Service. 2010:45-48.
［13］杨敬锋,张南峰,李勇,等. 基于改进Huffman编码的农机作业数据传输压缩方法［J］. 农业工程学报, 2014,30(13):153-159.
［14］居美艳,葛欣,李岳衡,等. 空时相关MIMO信道下的空时联合Huffman有限反馈预编码［J］. 通信学报, 2013,34(11):179-183.
［15］陈疆,郭克华,梁琳. 一类新的网络游戏场景3D障碍信息表达方案［J］. 中南大学学报(自然科学版), 2012,43(7):2636-2641.
［16］周桂宇,马宪民,李卫斌. 改进的混合压缩算法在GPS数据压缩中的应用［J］. 计算机应用与软件, 2013,30(12):167-169.
［17］洪旭升,林世平. 基于MapFile的HDFS小文件存储效率问题［J］. 计算机系统应用, 2012,21(11):179-182.

[1]	ZHONG Song-ying. Textile Raw Material Cost Warning Based on Apriori Algorithm of Association Rules [J]. Computer and Modernization, 2023, 0(07): 43-43.
[2]	CHEN Hao, ZHANG Ya, LUO Xi-chang, ZHANG Ya-li, LIU Wen-jing. Meteorological Data Storage and Retrieval System Based on MongoDB [J]. Computer and Modernization, 2020, 0(08): 100-104.
[3]	SU Lin-ping, AN Ran, LI Wei, CUI Wen-chao, ZHANG Xiao-liang. Design of Power Operation and Maintenance Audit System Based on Hadoop [J]. Computer and Modernization, 2020, 0(01): 49-.
[4]	MOCheng-wei,FANBing-bing. DOP：ASimpleOpenDataFrameworkandItsApplication [J]. Computer and Modernization, 2018, 0(07): 6-.
[5]	JIPeng1,2,LIHui1,2,CHENMei1,2,DAIZhen-yu1,2. DoFFT:AFastFourierTransformMethodBasedonDistributedDatabase [J]. Computer and Modernization, 2018, 0(06): 19-.
[6]	CHEN Lijuan, XIE Huosheng. A Parallel Algorithm for Mining onshelf Utility Itemset with Negative Item Values [J]. Computer and Modernization, 2018, 0(04): 13-.
[7]	LIU Sai1, NIE Qing-jie1, LIU Jun1, LI Dong-min2, LI Jing2. A New Access Control Model for Real Time Database #br# Backup System Based on Quantified Action [J]. Computer and Modernization, 2018, 0(01): 116-122.
[8]	YUAN Zhao-zheng1, SHAO Xiu-li1, YAN Kai-jing2, LI Dan2, GUO Jian-jun3. Design and Implementation of HBase Query Based on SQL [J]. Computer and Modernization, 2017, 0(7): 20-26+61.
[9]	LI Jin-yang, JIANG Shun-qing. Continuous Probabilistic Skyline Queries for Moving Data Points in Manhattan Road Networks [J]. Computer and Modernization, 2017, 0(7): 85-90.
[10]	SUN Xiao-yin, ZHOU Wei. Big Data Analytics Technology Based on MOOC [J]. Computer and Modernization, 2017, 0(4): 89-93,108.
[11]	DAI Sheng, WANG Bo. Relational Query in Large-scale Configuration Management Database Based on Graph Database [J]. Computer and Modernization, 2017, 0(1): 51-56.
[12]	ZHANG Chao1,2, GUO Hui-ming2, ZHANG Hong2. Cloud Storage Research on Supervisory Video Heat Based on K-means [J]. Computer and Modernization, 2016, 0(6): 12-15.
[13]	ZHU Qian1, QIN Hua1, FENG Zhi-xin2, CHEN Chen1. A Large Margin Nearest Neighbor Algorithm of Large-scale Text Classification [J]. Computer and Modernization, 2016, 0(6): 68-72.
[14]	WU Fei, MAO Yu-guang. A Multi-null Value Estimation Method Based on Multi-table Relationship Information in Relational Database [J]. Computer and Modernization, 2016, 0(6): 117-122.
[15]	CAO Gui-ning, SHI Cheng-wei, CHEN Xiang-xian. Service-oriented Cross-platform Database Data Sharing [J]. Computer and Modernization, 2016, 0(1): 94-103.