esearch and Application on Association Rule Mining Optimization Algorithm for High Speed EMU Malfunction

doi:10.3969/j.issn.1006-2475.2017.09.014

Abstract

Abstract: As high-speed and safe railway transportation, EMU is hard to avoid malfunction. In the view of the characteristics of EMU malfunction data—huge volume and low value density, we design an improved association rules mining algorithm based on the DHP algorithm. We adopt rehashing technique to solve the hash collision in the DHP algorithm and propose the RDHP algorithm, which can filter out all the infrequent item sets without additional database scanning. In order to enhance the efficiency of the algorithm, we propose the MR-RDHP algorithm based on the MapReduce programming framework. It decomposes the task of EMU malfunction data association rule mining to massive parallel processing on multiple computers in the cluster. The experimental result shows that the MR-RDHP algorithm has a good time performance and the rules dug out can be effectively used to guide EMU operation and maintenance.

Key words: EMU, association rules mining, DHP algorithm, rehashing, MapReduce

CLC Number:

TP391

ZHANG Chun, ZHOU Jing. esearch and Application on Association Rule Mining Optimization Algorithm for High Speed EMU Malfunction[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2017.09.014.

References

［1］ Agrawal R, Swami A. Mining association rules between sets of items in large databases［J］. ACM SIGMOD Record, 1993,22(2):207-216.
［2］ Park J S, Chen M S, Yu P S. An effective hash-based algorithm for mining association rules［J］. ACM SIGMOD Record, 1995,24(2):175-186.
［3］吴恒,吴根秀,毛临川,等. 一种基于DHP的动态链地址关联规则挖掘算法［J］. 江西师范大学学报(自然科学版), 2015(5):463-468.
［4］张忠林,田苗凤,刘宗成. 大数据环境下关联规则并行分层挖掘算法研究［J］. 计算机科学, 2016,43(1):286-289.
［5］徐慧,孙琪,杨林,等. 面向大数据的关联规则挖掘算法并行化设计［J］. 计算机科学, 2014,10(41):434-437.
［6］卢云彬,曹汉强. 基于Hash表的关联规则挖掘算法的改进［J］. 计算机技术与发展, 2007,17(6):12-14.
［7］王娟. 一种基于DHP算法的频繁项集改进方法［J］. 科技视界, 2013(31):47-48.
［8］张江,傅鹤岗. 基于关联规则的二维哈希算法的改进［J］. 计算机工程与设计, 2005,26(8):2178-2179.
［9］郭进伟,皮建勇. 基于MapReduce的SON算法实现［J］. 计算机应用, 2014,34(S1):100-102.
［10］胡辉,张骏温. 云环境下动车组故障数据挖掘研究与应用［J］. 计算机科学, 2014,41(10):413-416.
［11］任钦正. 动车组主动运维服务状态监测与故障诊断技术研究与应用［D］. 北京:北京交通大学, 2014.
［12］王爱平,王占凤,陶嗣干,等. 数据挖掘中常用关联规则挖掘算法［J］. 计算机技术与发展, 2010,20(4):105-108.
［13］杨秀萍. 大数据下关联规则算法的改进及应用［J］. 计算机与现代化, 2014(12):23-26.
［14］潘燕燕. 关联规则DHP算法的研究与分析［J］. 佛山科学技术学院学报(自然科学版), 2012,30(2):30-32.
［15］ Sonawani S, Mishra A. DHPTID-HYBRID Algorithm: A hybrid algorithm for association rule mining［C］// Proceedings of the 6th International Conference on Advanced Data Mining and Applications: Part I. 2010:149-160.
［16］ Najadat H, Shatnawi A, Obiedat G. A new perfect hashing and pruning algorithm for mining association rule［J］. Communications of the Ibima, 2011,2011(652178):4715-4725.
［17］ Lee Hyung-Bong, Kwon Ki-Hyeon. An optimization of hashing mechanism for the DHP association rules mining algorithm［J］. Journal of The Korea Society of Computer and Information, 2010,15(8)13-21.

[1]	SHI Hui1, CHEN En2. Distributed Stage Adaptive Association Rules Mining Algorithm Based on Spark [J]. Computer and Modernization, 2019, 0(12): 31-.
[2]	SONG Yang, SHI Hong-yan. Improved K-means Clustering Algorithm Based on MapReduce Framework [J]. Computer and Modernization, 2019, 0(08): 28-.
[3]	CHEN Lijuan, XIE Huosheng. A Parallel Algorithm for Mining onshelf Utility Itemset with Negative Item Values [J]. Computer and Modernization, 2018, 0(04): 13-.
[4]	YI Xiu-liang1, REN Shu-ting2, SUN Yan-tao1,3. A Mobile Ad-hoc Network Testbed Based on Ethernet [J]. Computer and Modernization, 2017, 0(7): 72-78.
[5]	LI Mu-gong, LIU Yu-ling, YIN Fang, ZHANG Yong-bo. Simulation Test Platform for EMU Braking System Based on Virtual Instrument [J]. Computer and Modernization, 2017, 0(5): 50-55.
[6]	ZHANG Chun, YUAN Tian-ning. Design of Multi-source and Heterogeneous Data Fusion Framework for Integrated Lifecycle Management of EMU [J]. Computer and Modernization, 2017, 0(10): 36-41.
[7]	LE Yi1,2. Application of Learning Analytics Technology in Big Data Environment [J]. Computer and Modernization, 2016, 0(9): 104-108.
[8]	XU Huan1, FENG Jun1, ZHANG Peng-cheng1, TANG Zhi-xian2, LIU Yi1, CHEN Zhi-fei1, ZHANG Li-xia1. Distributed CIF Quadtree Indexing Method Based on Hadoop [J]. Computer and Modernization, 2016, 0(11): 12-19,24.
[9]	CAI Ming， ZUO Yongan. A Hybrid Ant Colony Algorithm Based on MapReduce [J]. Computer and Modernization, 2016, 0(10): 6-.
[10]	XU Yuanbin. Energysaving Optimization Model Under Big Data Environment #br# Based on Genetic Algorithm [J]. Computer and Modernization, 2016, 0(10): 25-29.
[11]	WANG Hao. Scheduling Strategy of Reduce Task Based on Data Localization in Hadoop [J]. Computer and Modernization, 2016, 0(1): 114-120.
[12]	HU Long, LUO Jun. Hybrid Join Algorithm Based on MapReduce [J]. Computer and Modernization, 2015, 0(6): 86-91.
[13]	LIU Shu-ying. A Top-k Closest Pairs Data Search Scheme Based on MapReduce [J]. Computer and Modernization, 2014, 0(8): 38-45.
[14]	LENG Yong-lin. Distributed AP Clustering Algorithm Based on MapReduce [J]. Computer and Modernization, 2014, 0(10): 104-107.
[15]	GU Rui-chun1, WANG Jing-yu2. A Parallel Clustering Model Based on MapReduce [J]. Computer and Modernization, 2014, 0(1): 90-92,108.