A Parallel Clustering Model Based on MapReduce

Abstract

Abstract: During the clustering for big scale data, the traditional serial model has limitations, can not obtain a satisfied results within significant intervals. This paper proposes a concurrent clustering model based on MapReduce architechture under the Hadoop platform. The experiment result shows this model has a perfect linear speedup, and it outperforms the traditional clusting model, especially in dealing with massive data set.

Key words: data mining, algorithm, cloud computing, MapReduce, Hadoop

GU Rui-chun1, WANG Jing-yu2. A Parallel Clustering Model Based on MapReduce[J]. Computer and Modernization, 2014, 0(1): 90-92,108.

References

［1］
刘洋. 基于MapReduce的中医药并行数据挖掘服务［D］. 杭州：浙江大学, 2010.
［2］李成华,张新访,金海,等. MapReduce:新型的分布式并行计算编程模型［J］. 计算机工程与科学, 2011,33(3):129-135.
［3］郝晓飞,谭跃生,王静宇. Hadoop平台上Apriori算法并行化研究与实现［J］. 计算机与现代化, 2013(3):1-5.
［4］施佺,肖仰华,温文灏,等. 基于Mapreduce的大规模社会网络提取方法研究［J］. 计算机应用研究, 2011,28(1):145-148.
［5］刘永增. 基于Hadoop/Hive的海量Web日志处理系统的设计与实现［D］. 大连：大连理工大学, 2011.
［6］舒琰,向阳,张骐,等. 基于PageRank的微博排名MapReduce算法研究［J］. 计算机技术与发展, 2013,23(2):73-76,81.
［7］张宇,程久军. 基于MapReduce的矩阵分解推荐算法研究［J］. 计算机科学, 2013,40(1):19-21,36.
［8］丁光华,周继鹏,周敏. 基于MapReduce的并行贝叶斯分类算法的设计与实现［J］. 微计算机信息, 2010,36(9):190-191,176.
［9］艾树宇. 基于Hadoop/MapReduce的K_NN算法［J］. 科技传播, 2013(1):203-204,200.
［10］孟海东,杨彦侃. 并行聚类算法的设计与研究［J］. 计算机与现代化, 2010(8):5-8.
［11］王淑玲. 增量聚类算法的设计与实现［D］. 包头：内蒙古科技大学, 2009.
［12］戎翔,李玲娟. 基于MapReduce的频繁项集挖掘方法［J］. 西安邮电学院学报, 2011,16(4):37-39,43.
［13］梁建武,周杨. 一种异构环境下的Hadoop调度算法［J］. 中国科技论文, 2012,7(7):495-498.
［14］李锐,王斌. 文本处理中的MapReduce技术［J］. 中文信息学报, 2012,26(4):9-20.
［15］周迎春,骆嘉伟. 一种改进的BIRCH聚类分析算法及其应用研究［J］. 湛江师范学院学报, 2009,30(3):83-87.

[1]	WANG Xiaohang1, LI Yongjie1, YU Lei1, FAN Xiao2. A Method of Using Compound Event Probability Operation to Solve Problem of Negative Information Blocking Maximization [J]. Computer and Modernization, 2024, 0(12): 24-33.
[2]	LYU Meijing1, NIAN Mei1, ZHANG Jun1, 2, FU Lusen1. Anomaly Detection of Network Traffic Based on Autoencoder [J]. Computer and Modernization, 2024, 0(12): 40-44.
[3]	YUAN Qingle, MU Li. Inventory Forecasting Method Based on Improved Elman Neural Network [J]. Computer and Modernization, 2024, 0(11): 28-33.
[4]	LI Junchao1, YOU Fei1, ZHANG Chao2, SU Lele2, GONG Yan2 . BiLSTM-Attention Prediction Model and Error Analysis #br# Based on Novel Multi-objective Coati Optimization Algorithm [J]. Computer and Modernization, 2024, 0(11): 70-76.
[5]	ZHANG Zhixia, QIN Zhiyi. Network Public Opinion Prediction Based on Variational Mode Decomposition and IGJO-SVR [J]. Computer and Modernization, 2024, 0(11): 77-83.
[6]	YANG Zhengke, SHEN Xiaodong, WANG Kaixiang, HE Li. Grounding Grid Corrosion Localization Based on Improved Sparrow Search Algorithm [J]. Computer and Modernization, 2024, 0(10): 14-20.
[7]	HUANG Shanshan1, WU Wei2, XU Yuqing1, WEI Jie1. Pipelines in Drawings Detection Method Based on Improved Mask R-CNN and LSD [J]. Computer and Modernization, 2024, 0(10): 42-48.
[8]	DU Mengjun1, LI Ang1, TONG Jun1, QIAN Jin1, KANG Kai1, WANG Ruoding1, JIN Wenxing2. Power Information Data Fusion Model Based on Improved Extreme Learning Algorithm [J]. Computer and Modernization, 2024, 0(10): 61-64.
[9]	WANG Jia1, GU Wenjun1, JU Weigang2, LI Yuwei1, ZHANG Yunlong2, MI Chuanmin3, ZHOU Zhipeng3. Environmental Topology Task Scheduling Based on Diverse Hierarchical Difference Optimization Genetic Algorithm [J]. Computer and Modernization, 2024, 0(10): 65-73.
[10]	YU Tianyi, LI Jianfeng, CHEN Hailong, ZHAI Jun. Collaborative Recommendation Algorithm with Implicit Roles [J]. Computer and Modernization, 2024, 0(09): 1-7.
[11]	ZHANG Huinan1, ZHANG Qiang1, SUN Hongxia2. Dynamic Analysis Model of Reservoir Production Based on Improved#br# Time-series Capsule Network [J]. Computer and Modernization, 2024, 0(09): 15-19.
[12]	YANG Yufeng1, 2, XIA Xiaoyun2, CHEN Zefeng3, LIAO Weizhi2, LI Jiwu2. Dung Beetle Optimization Algorithm Integrating Multiple Strategies for Take-out Order Distribution Route Optimization [J]. Computer and Modernization, 2024, 0(09): 25-32.
[13]	XU Xiaowei, CHENG Yu, QIAN Feng, ZHU Neng, DENG Mingxing. Cryptographic Algorithm of IoV Communication Based on AES [J]. Computer and Modernization, 2024, 0(09): 45-51.
[14]	JIANG Xinzi1, AN Xiaoli1, GAO Shang2. Smart Delivery Service of Public Libraries Based on MTSP Problem [J]. Computer and Modernization, 2024, 0(09): 52-55.
[15]	GAO Meng, ZENG Xianwen. Improved Pelican Optimization Algorithm Based on Circle Mapping and#br# Adaptive t-Distribution Mutation [J]. Computer and Modernization, 2024, 0(09): 69-73.