Semantic Retrieval Method for Water Conservancy Metadata Based on Hadoop

doi:10.3969/j.issn.1006-2475.2015.12.001

Abstract

Abstract: In order to provide a solution for the absence of semantic comprehension of metadata search engine in water conservancy domain together with the problem of low efficiency when indexing water conservancy metadata, a semantic retrieval method for water conservancy metadata based on Hadoop is brought forward in this paper. First, the semantic searching method with the combination of ontology and query expansion technology is used to design ontology reasoning rules, semantic similarity calculation method, expansion words selecting method and semantic relevance ordering method so as to effectively improve the recall ratio and precision ratio of search results. Second, as for the problem of low efficiency when building an index of water conservancy metadata in XML form, MapReduce parallel processing model in Hadoop platform is introduced to make parallel processing, analysis and extraction of metadata information and index building, and to modify the file structure of SequenceFile in response to the small files of water conservancy metadata and performance bottleneck of water conservancy metadata index building under centralized environment. Finally, semantic extension query method is designed by using of the powerful parallel computing capability of Hadoop so as to improve the query efficiency of water conservancy metadata.

Key words: Key words: domain ontology, calculation of similarity, semantic query, Hadoop, SequenceFile

CLC Number:

TP391

FENG Jun， LI Zongxiang， TANG Zhixian， JIANG Kang. Semantic Retrieval Method for Water Conservancy Metadata Based on Hadoop[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2015.12.001.

References ［17］

	［1］
	水利部水利信息化工作领导小组办公室. 2012年度中国水利信息化发展报告［M］. 北京:中国水利水电出版社, 2013.
［2］	冯钧,唐志贤,朱跃龙,等. 水利信息资源目录服务元数据定义研究［J］. 水利信息化, 2011(S1):19-22.
［3］	Guha R, McCool R, Miller E. Semantic search［C］// Proceedings of the 12th International Conference on World Wide Web. ACM, 2003: 700-709.
［4］	Yu Xuejun, Lv Jing. Keywords semantic extension in semantic search model［C］// International Conference on Computer, Networks and Communication Engineering (ICCNCE 2013). Atlantis Press, 2013.
［5］	Armentano M G, Godoy D, Campo M, et al. NLPbased faceted search: Experience in the development of a science and technology search engine［J］. Expert Systems with Applications, 2014,41(6):2886-2896.
［6］	Tran T, Cimiano P, Rudolph S, et al. Ontologybased interpretation of keywords for semantic search［M］// The Semantic Web. Springer Berlin Heidelberg, 2007:523-536.
［7］	Qiu Yonggang, Frei H P. Concept based query expansion［C］// Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1993:160-169.
［8］	Attar R, Fraenkel A S. Local feedback in fulltext retrieval systems［J］. Journal of the ACM (JACM), 1977,24(3):397-417.
［9］	Mena E, Illarramendi A, Kashyap V, et al. OBSERVER: An approach for query processing in global information systems based on interoperation across preexisting ontologies［J］. Distributed and Parallel Databases, 2000,8(2):223-271.
［10］	Khatchadourian S, Consens M P, Siméon J. Having a ChuQL at XML on the Cloud［C］// A. Mendelzon International Workshop(AMW). 2011.
［11］	Shah B, Rao P R, Moon B, et al. A data parallel algorithm for XML DOM parsing［M］// Database and XML Technologies. Springer Berlin Heidelberg, 2009:75-90.
［12］	魏永山,张峰,陈欣,等. 一种云计算环境下的XML查询数据服务的优化方法［J］. 计算机工程与科学, 2013,35(6):30-36.
［13］	Rada R, Mili H, Bichnell E, et al. Development and application of a metricon semantic nets［J］. IEEE Transactions on Systems, Man and Cybernetics, 1989,9(1):17-30.
［14］	Wu Zhibiao, Palmer M. Verb semantics and lexical selection［C］// Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics. 1994:133-138.
［15］	韩欣,樊永生,马春森,等. 基于树状结构的语义相似度计算方法分析［J］. 微电子学与计算机, 2012,29(5):38-41.
［16］	Choi I, Kim M. Topic distillation using hierarchy concept tree［C］// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2003:371-372.
［17］	曹叡,吴玲达. 一种改进的领域本体语义相似度计算方法［J］. 微电子学与计算机, 2014,31(8):109-114.

[1]	ZHANG Jun, SU Wen-hao. Optimization Method of Hadoop File Archiving Based on LZO [J]. Computer and Modernization, 2023, 0(06): 1-6.
[2]	SU Lin-ping, AN Ran, LI Wei, CUI Wen-chao, ZHANG Xiao-liang. Design of Power Operation and Maintenance Audit System Based on Hadoop [J]. Computer and Modernization, 2020, 0(01): 49-.
[3]	WANG Liang-yu. Knowledge Graph Construction Method for Carbon Trading [J]. Computer and Modernization, 2018, 0(08): 114-.
[4]	ZHOU Tian-qi. Spatial Distribution Statistics of Urban Population Based on Mobile Communication Big Data [J]. Computer and Modernization, 2018, 0(05): 45-.
[5]	WEN He-ping1, BAO Jing-jing2, KE Ju-xin1, LIU Shu-wei1. Design of Hyperchaotic Lorenz Encryption Algorithm Based on Hadoop [J]. Computer and Modernization, 2018, 0(03): 108-.
[6]	ZHU Shan, AI Li-hua. Large-scale Image Storage and Retrieval Based on Hadoop [J]. Computer and Modernization, 2017, 0(6): 61-66+83.
[7]	LI Cheng, CHAI Xiao-li, XIE Bin, TANG Peng. A Resource Scheduling Mechanism of Hadoop YARN [J]. Computer and Modernization, 2017, 0(11): 29-34.
[8]	ZHANG Jin, FENG Jun, LU Jia-min. Spatial Keyword Index Method Based on Hadoop [J]. Computer and Modernization, 2017, 0(11): 76-83.
[9]	FENG Jun, XU Wei-gang, FENG Du-qing, LU Jia-min, XU Huan. Research on Index Method of Massive Hydrology Data [J]. Computer and Modernization, 2017, 0(10): 29-35,41.
[10]	LI Na, CHEN Zheng-ming, LYU Jia, LIU Chun-fang. Transaction Design and Implementation for HDFS Access Middleware [J]. Computer and Modernization, 2017, 0(1): 46-50.
[11]	SUN Li-hua1, HU Mu1, MENG Qing-qiang1, QIAN Ya-kang1, WONG Song2. Solution for High Performance of Distribution Network Line Loss Big Data [J]. Computer and Modernization, 2016, 0(12): 42-46,50.
[12]	XU Huan1, FENG Jun1, ZHANG Peng-cheng1, TANG Zhi-xian2, LIU Yi1, CHEN Zhi-fei1, ZHANG Li-xia1. Distributed CIF Quadtree Indexing Method Based on Hadoop [J]. Computer and Modernization, 2016, 0(11): 12-19,24.
[13]	QI Peng-nian, ZHU Jin, HAO Jun-hui, XU Feng-ping. Hadoop Speculation Execution Algorithm in Heterogeneous Environments [J]. Computer and Modernization, 2015, 0(8): 80-83,88.
[14]	HU Long, LUO Jun. Hybrid Join Algorithm Based on MapReduce [J]. Computer and Modernization, 2015, 0(6): 86-91.
[15]	WANG Bin, LIU Zhao-yuan. Optimization Model of Remote File Synchronization Based on Rsync Algorithm [J]. Computer and Modernization, 2015, 0(4): 10-13.