基于Hadoop的空间关键字索引方法

doi:10.3969/j.issn.1006-2475.2017.11.015

计算机与现代化

基于Hadoop的空间关键字索引方法

(河海大学计算机与信息学院，江苏南京 211100)

收稿日期:2017-03-09 出版日期:2017-11-21 发布日期:2017-11-21
作者简介:张进(1992-)，男，安徽芜湖人，河海大学计算机与信息学院硕士研究生，研究方向：时空间数据管理，云计算; 冯钧(1969-)，女，教授，博士，研究方向：时空间数据管理，智能数据处理与数据挖掘，水利信息化; 陆佳民(1983-)，男，讲师，研究方向：移动对象数据管理，分布式数据处理。
基金资助:
国家自然科学基金资助项目(61370091, 61602151)

Spatial Keyword Index Method Based on Hadoop

(College of Computer and Information, Hohai University, Nanjing 211100, China)

Received:2017-03-09 Online:2017-11-21 Published:2017-11-21

摘要/Abstract

摘要： 随着移动互联网的兴起，在移动端产生了海量的数据，这些数据除了含有传统的文本信息外还含有空间位置信息。为了能够高效地处理和利用这些数据，高效的空间关键字索引方法已经成为业界研究的热点。但现有的空间关键字索引方法，在面对海量空间数据时还存在缺乏伸缩性和可扩展性、易产生查询热点等问题。为此，本文提出一种基于Hadoop的空间关键字索引方法，并对索引并行化算法进行优化，与已有的空间关键字索引方法进行对比。实验结果验证了本文提出的空间关键字索引方法的有效性。

关键词: 空间关键字, 索引, 分布式查询, Hadoop

Abstract: With the rise of the mobile Internet, huge amounts of data are generated from mobile terminal, these data contain not only traditional text information but also spatial location information. In order to effectively process and utilize these data, highly efficient methods of spatial keyword index have become a hot research topic in the field. But in the face of huge amounts of spatial data, the existing methods of spatial keyword index still have problems such as the lack of scalability and extensibility, easy to generate query hotspots. In order to deal with the above problems, two improvements measures are proposed in this paper: the method of spatial keyword index based on Hadoop and optimized parallelizable algorithm of index. Finally, we compare the spatial keyword index method proposed in this paper with the existing spatial keyword index methods and verify its effectiveness.

Key words: spatial keyword, index, distributed query, Hadoop

中图分类号:

TP301.6

张进，冯钧，陆佳民. 基于Hadoop的空间关键字索引方法[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2017.11.015.

ZHANG Jin, FENG Jun, LU Jia-min. Spatial Keyword Index Method Based on Hadoop[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2017.11.015.

参考文献

[1] 周傲英,杨彬,金澈清,等. 基于位置的服务:架构与进展[J]. 计算机学报, 2011,34(7):1155-1171.

[2] Li Guoliang, Hu Jin, Feng Jianhua, et al. Effective location identification from microblogs[C]// Proceedings of the 30th IEEE International Conference on Data Engineering (ICDE). 2014:880-891.

[3] Cao Xin, Chen Lisi, Cong Gao, et al. Spatial keyword querying[C]// Proceedings of the 31st International Conference on Conceptual Modeling. 2012:16-29.

[4] Chen Lisi, Cong Gao, Jensen C S, et al. Spatial keyword query processing: An experimental evaluation[J]. Proceedings of the VLDB Endowment, 2013,6(3):217-228.

[5] Luo Siqiang, Luo Yifeng, Zhou Shuigeng, et al. Distributed spatial keyword querying on road networks[C]// Proceedings of the 17th International Conference on Extending Database Technology. 2014:235-246.

[6] De Felipe I, Hristidis V, Rishe N. Keyword search on spatial databases[C]// Proceedings of the 24th IEEE International Conference on Data Engineering. 2008:656-665.

[7] Guttman A. R-trees: A dynamic index structure for spatial searching[C]// Proceedings of the 1984 ACM SIGMOD International Conference on Management of Data. 1984:47-57.

[8] Tao Yufei, Papadias D, Shen Qiongmao. Continuous nearest neighbor search[C]// Proceedings of the 28th International Conference on Very Large Databases. 2002:287-298.

[9] 邵奇峰,李枫. 一种基于HBase的空间关键字查询算法[J]. 计算机工程与科学, 2015,37(11):2084-2090.

[10] Zhou Yinghua, Xie Xing, Wang Chuang, et al. Hybrid index structures for location-based Web search[C]// Proceedings of the 14th ACM International Conference on Information and Knowledge Management. 2005:155-162.

[11] Wu Dingming, Yiu M L, Cong Gao, et al. Joint top-k spatial keyword query processing[J]. IEEE Transactions on Knowledge and Data Engineering, 2012,24(10):1889-1903.

[12] Vaid S, Jones C B, Joho H, et al. Spatio-textual indexing for geographical search on the Web[C]// Proceedings of the 9th International Conference on Advances in Spatial and Temporal Databases. 2005:218-235.

[13] Tao Yufei, Sheng Cheng. Fast nearest neighbor search with keywords[J]. IEEE Transactions on Knowledge and Data Engineering, 2014,26(4):878-888.

[14] Samet H. The quadtree and related hierarchical data structures[J]. ACM Computing Surveys, 1984,16(2):187-260.

[15] Wu Sai, Wu Kun-Lung. An indexing framework for efficient retrieval on the cloud[J]. IEEE Data Engineering Bulletin, 2009,32(1):75-82.

[16] Zhou Xin, Zhang Xiao, Wang Yanhao, et al. Efficient distributed multi-dimensional index for big data management[C]// Proceedings of the 14th International Conference on Web-Age Information Management. 2013:130-141.

[17] Zou Yongqiang, Liu Jia, Wang Shicai, et al. CCIndex: A complemental clustering index on distributed ordered tables for multi-dimensional range queries[C]// Proceedings of the 2010 IFIP International Conference on Network and Parallel Computing. 2010:247-261.

[18] Dittrich J, Quiané-Ruiz J A, Richter S, et al. Only aggressive elephants are fast elephants[J]. Proceedings of the VLDB Endowment, 2012,5(11):1591-1602.

[19] 马友忠,孟小峰.云数据管理索引技术研究[J]. 软件学报, 2015,26(1):145-166.

[20] Chang W W, Schek H J. A signature access method for the Starburst database system[C]// Proceedings of the 15th International Conference on Very Large Databases. 1989:145-153.

[21] Gremillion L L. Designing a Bloom filter for differential file access[J]. Communications of the ACM, 1982,25(9):600-604.

[22] Orenstein J A, Merrett T H. A class of data structures for associative searching[C]// Proceedings of the 3rd ACM SIGACT-SIGMOD Symposium on Principles of Database Systems. 1984:181-190.

[23] Kirsch A, Mitzenmacher M. Less hashing, same performance: Building a better Bloom filter[C]// Proceedings of the 14th Conference on Annual European Symposium. 2006,14:456-467.

基于Hadoop的空间关键字索引方法

Spatial Keyword Index Method Based on Hadoop

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张军, 苏文浩 . 基于LZO的Hadoop文件归档优化方法[J]. 计算机与现代化, 2023, 0(06): 1-6.
[2]	邱金水, 庄会富, 金涛. 面向海量植物图像的智能检索系统设计[J]. 计算机与现代化, 2022, 0(10): 62-67.
[3]	许贤慧, 王淑营, 曾文驱. 面向工程数据检索的ElasticSearch索引优化策略[J]. 计算机与现代化, 2022, 0(02): 79-84.
[4]	徐惠, 铁治欣, 舒莹. 基于特征匹配的快速降维排序搜索方法[J]. 计算机与现代化, 2022, 0(02): 85-91.
[5]	苏林萍，安然，李为，崔文超，张晓良. 基于Hadoop的电力运维审计系统的设计[J]. 计算机与现代化, 2020, 0(01): 49-.
[6]	梁珺秀,许建秋. 基于时空标签轨迹的范围模式匹配查询[J]. 计算机与现代化, 2018, 0(08): 79-.
[7]	周天绮. 基于移动通信大数据的城市人口空间分布统计[J]. 计算机与现代化, 2018, 0(05): 45-.
[8]	吴克介,王家伟. 基于知网与搜索引擎的词汇语义相似度计算[J]. 计算机与现代化, 2018, 0(04): 90-.
[9]	温贺平1,鲍晶晶2,柯居鑫1,刘树威1. 基于Hadoop的Lorenz超混沌加密算法设计[J]. 计算机与现代化, 2018, 0(03): 108-.
[10]	字云飞,李业丽,孙华艳,张莉婧. 改进FP-Growth算法在旅游线路规划中的应用研究[J]. 计算机与现代化, 2018, 0(02): 17-.
[11]	李俊廷，瞿有利. 基于分区的Elias-Fano-Golomb-Rice倒排索引压缩算法[J]. 计算机与现代化, 2017, 0(9): 33-39.
[12]	高菲. 基于P2P的浏览器缓存协作系统[J]. 计算机与现代化, 2017, 0(8): 98-.
[13]	朱珊，艾丽华. 基于Hadoop的大规模图像存储与检索[J]. 计算机与现代化, 2017, 0(6): 61-66+83.
[14]	李全,林松,田俊,刘兴红. 面向MOOC的垂直搜索引擎[J]. 计算机与现代化, 2017, 0(4): 32-37.
[15]	李程，柴小丽，谢彬，唐鹏. 一种Hadoop YARN的资源调度机制[J]. 计算机与现代化, 2017, 0(11): 29-34.