基Word Embedding的软件工程领域语义相关词挖掘方法

doi:10.3969/j.issn.1006-2475.2017.09.004

计算机与现代化

基Word Embedding的软件工程领域语义相关词挖掘方法

上海交通大学软件学院，上海 200240

收稿日期:2016-12-01 出版日期:2017-09-20 发布日期:2017-09-19
作者简介:胡望胜（1993-），男，湖南汨罗人，上海交通大学软件学院硕士研究生，研究方向：程序分析与测试。
基金资助:
国家自然科学基金资助项目(61572312,61572313)；上海市科委科研项目(15DZ1100305)

Learning Semantically Related Words in Software Through Word Embedding

School of Software, Shanghai Jiao Tong University, Shanghai 200240, China

Received:2016-12-01 Online:2017-09-20 Published:2017-09-19

摘要/Abstract

摘要： 软件的开发及维护过程中经常要对代码进行搜索。基于关键字匹配的代码搜索面临与传统信息检索一样的问题，即用户查询关键字与代码文本用词不匹配。为提高代码搜索精度，需要挖掘软件中的语义相关词进行查询扩展。本文针对软件工程领域设计了一种基于Word Embedding的语义相关词挖掘方法，并且采用IT技术问答网站Stack Overflow的文档作为语料库训练得到了共包含19332个单词的语义相关词表。与前人工作的对比实验验证了本文方法挖掘的语义相关词能有效提高代码搜索精度。

关键词: 代码搜索, 查询扩展, 语义相关词

Abstract: Searching for previously written code is important for software development and maintenance. The same as traditional information retrieval, the inherent difficulty of keyword based code search is vocabulary mismatch between user query and retrieved code. To improve the accuracy of code search, learning semantically related words in software for query expansion is needed. This paper designs a Word Embedding based method to learn semantically related words in software, and obtains semantically related words for 19332 words through training it on Stack Overflow documents. The experiment results show that the learned semantically related words can effectively improve code search accuracy.

Key words: code search, query expansion, semantically related words

中图分类号:

TP311

胡望胜. 基Word Embedding的软件工程领域语义相关词挖掘方法[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2017.09.004.

HU Wang-sheng. Learning Semantically Related Words in Software Through Word Embedding[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2017.09.004.

参考文献

［1］ Shepherd D, Damevski K, Ropski B, et al. Sando: An extensible local code search framework［C］// Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. 2012:1-2.
［2］ Furnas G W, Landauer T K, Gomez L M, et al. The vocabulary problem in human-system communication［J］. Communications of the ACM, 1987,30(11):964-971.
［3］ Shepherd D, Fry Z P, Hill E, et al. Using natural language program analysis to locate and understand action-oriented concerns［C］// Proceedings of the 6th International Conference on Aspect-oriented Software Development. 2007:212-224.
［4］ Hill E. Integrating Natural Language and Program Structure Information to Improve Software Search and Exploration［D］. University of Delaware, 2010.
［5］ Yang Jinqiu, Lin Tan. Inferring semantically related words from software context［C］// Proceedings of the 9th IEEE Working Conference on Mining Software Repositories. 2012:161-170.
［6］ Yang Jinqiu, Lin Tan. Swordnet: Inferring semantically related words from software context［J］. Empirical Software Engineering, 2014,19(6):1856-1886.
［7］ Tian Yuan, Lo D, Lawall J. Sewordsim: Software-specific word similarity database［C］// Proceedings of the 36th International Conference on Software Engineering. 2014:568-571.
［8］ Sando. Sando Code Search Tool［EB/OL］. http://sandosearch.weebly.com/, 2016-12-01.
［9］ Aragon Consulting Group. Krugle Code Search［EB/OL］. http://www.krugle.com/, 2016-12-01.
［10］ Linstead E, Bajracharya S, Ngo T, et al. Sourcerer: Mining and searching internet-scale software repositories［J］. Data Mining and Knowledge Discovery, 2009,18(2):300-336.
［11］ Merriam-Webster. Merriam-Webster English Dictionary and Thesaurus［EB/OL］. http://www.merriam-webster.com/, 2016-12-01.
［12］ Princeton University. WordNet［EB/OL］. http://wordnet.princeton.edu/, 2016-12-01.
［13］ Sridhara G, Hill E, Pollock L, et al. Identifying word relations in software: A comparative study of semantic similarity tools［C］// Proceedings of the 16th IEEE International Conference on Program Comprehension. 2008:123-132.
［14］ Turian J, Ratinov L, Bengio Y. Word representations: A simple and general method for semi-supervised learning［C］// Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 2010:384-394.
［15］ Mikolov T, Chen Kai, Corrado G, et al. Efficient estimation of word representations in vector space［C］// Proceedings of the 1st International Conference on Learning Representations. 2013.
［16］ Martin Porter. The Porter Stemming Algorithm［EB/OL］. https://tartarus.org/martin/PorterStemmer/, 2016-12-01.
［17］ Word2vec. Tool for Computing Continuous Distributed Representations of Words［EB/OL］. https://code.google.com/archive/p/word2vec/, 2016-12-01.
［18］ Shepherd D. Action-oriented Concerns［EB/OL］. https://www.eecis.udel.edu/~gibson/context/action_oriented_concerns.txt, 2016-12-01.

[1]	王千千，罗文兵. 基于词重要性的Markov网络查询扩展模型[J]. 计算机与现代化, 2017, 0(11): 89-94.
[2]	吕飞. 一种高效的源代码搜索算法[J]. 计算机与现代化, 2015, 0(3): 9-14.
[3]	冯钧，许潇，唐志贤，卞一路. 面向水利信息资源目录服务的分布式语义检索方法研究[J]. 计算机与现代化, 2015, 0(2): 122-.
[4]	田永昌1，李颖2. 基于兴趣模型的查询扩展[J]. 计算机与现代化, 2014, 0(7): 36-39.
[5]	王曼;吴振忠. 基于领域本体的语义搜索——带权最短路径方法[J]. 计算机与现代化, 2013, 1(9): 1-7.
[6]	黄名选;冯平;谢统义. 基于频繁项集挖掘与查询扩展的信息检索系统模型[J]. 计算机与现代化, 2012, 1(200): 53-03.
[7]	曾庆鹏吴水秀. 网络教学平台中问答系统的关键技术研究 [J]. 计算机与现代化, 2010, 1(7): 23-26.

基Word Embedding的软件工程领域语义相关词挖掘方法

Learning Semantically Related Words in Software Through Word Embedding

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价