计算机与现代化

• 数据库与数据挖掘 • 上一篇    下一篇

大数据复杂事件分析方法研究与应用

  

  1. (北方工业大学计算机学院,北京100144)
  • 收稿日期:2018-01-24 出版日期:2018-09-11 发布日期:2018-09-11
  • 作者简介:赵会群(1960-),男,辽宁沈阳人,北方工业大学计算机学院教授,博士,研究方向:软件工程,大数据; 乔玉衡(1992-),男,天津蓟县人,硕士研究生,研究方向:大数据处理。
  • 基金资助:
    国家自然科学基金资助项目(61672041)

Research and Application of Big Data Complex Event Pattern

  1. (School of Computer, North China University of Technology, Beijing 100144, China)
  • Received:2018-01-24 Online:2018-09-11 Published:2018-09-11

摘要: 复杂事件处理(Complex Event Processing, CEP)是一项伴随流式数据而出现的技术,用于不同数据源顺序混杂的事件流中发现感兴趣的事件模式。然而,随着数据量的不断递增,传统的CEP技术往往不能满足在大数据集上有效获取事件模式的处理需求。针对这一问题,本文结合数据挖掘中聚类分析与关联规则的思想,提出一种“复杂事件处理”算法,〖JP2〗并把其部署到分布式平台Hadoop上,从而发现大数据集中的复杂事件关系,有效地改变了传统技术面临海量数据的局限性。最后,应用本文算法到GPS大数据集中,发现其中的复杂事件模式,并通过实验验证本文方法具有可行性与有效性。

关键词: 复杂事件, 事件模式, 大数据, 聚类分析, 关联规则

Abstract: Complex Event Processing (CEP) is a technical method that comes with streaming data. It is used to find interesting event patterns in event streams that are mixed in different data sources. However, with the increasing amount of data, the traditional CEP techniques often fail to address the processing needs for efficient access to event patterns on big data sets. In response to this problem, this paper combines the idea of cluster analysis and association rules in data mining, proposes a “complex event processing” algorithm, and deploys it to the distributed platform Hadoop, thereby discovering the relationship between complex events in 〖JP2〗big data sets and effectively changing the limitations of traditional technologies facing massive data. Finally, the algorithm is applied to GPS big data set and the complex event patterns are found out. Experiments show that the method is feasible and effective.

Key words: complex event, event pattern, big data, cluster analysis, association rule

中图分类号: