Text Similarity Calculation Method Based on Levenshtein and TFRSF

doi:10.3969/j.issn.10062475.2018.04.016

Abstract

Abstract: Finding and collecting personal information in social networks can establish the information system with the curriculum vitae, life, hobbies, friends and the other attributes. But there are lots of same name users in different social networks. In order to solve the ambiguity of the same name, we calculate the user information similarity to decide whether they belong to the same person. If the information describing the document position is reversed, it will lead to computer misjudgment. In order to solve this problem, the Levenshtein and TFRSF methods are used to calculate the word frequency and edit distance to judge whether the attribute values are the same. The experimental results show that the proposed method of calculating the similarity of texts improves the accuracy of various evaluation indexes. The precision, recall and F1 of this method are more than 87%.

Key words: personal information, social network, Levenshtein, TFRSF, similarity

CLC Number:

TP391

ZANG Runqiang1, SUN Hongguang1,2, YANG Fengqin1,2, FENG Guozhong1,2, YIN Liangliang1. Text Similarity Calculation Method Based on Levenshtein and TFRSF[J]. Computer and Modernization, doi: 10.3969/j.issn.10062475.2018.04.016.

References

［1］ Gomaa W H, Fahmy A A. A survey of text similarity approaches［J］. International Journal of Computer Applications, 2013,68(13):1318.
［2］ Kwon O, Wen Yixing. An empirical study of the factors affecting social network service use［J］. Computers in Human Behavior, 2010,26(2):254263.
［3］ Levenshtein V I. Binary codes capable of correcting spurious insertions and deletions of ones［J］. Problems of Information Transmission, 1965,1(1):817.
［4］刘辉平,金澈清,周傲英. 一种基于模式的实体解析算法［J］. 计算机学报, 2015,38(9):17961808.
［5］ Li Yujian, Liu Bo. A normalized Levenshtein distance metric［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007,29(6):10911095.
［6］ Putra M E W, Suwardi I S. Structural offline handwriting character recognition using approximate subgraph matching and Levenshtein distance［J］. Procedia Computer Science, 2015,59:340349.
［7］邵清,叶琨. 基于编辑距离和相似度改进的汉字字符串匹配［J］. 电子科技, 2016,29(9):711.
［8］张润梁,牛之贤. 基于基本操作序列的编辑距离顺序验证［J］. 计算机科学, 2016,43(S1):5154.
［9］ Popescu D A, Nicolae D. Determining the similarity of two Web applications using the edit distance［C］// Proceedings of the 6th International Workshop on Soft Computing Applications. 2014,1:681690.
［10］Crochemore M, Iliopoulos C S, Langiu A, et al. The longest common substring problem［J］. Mathematical Structures in Computer Science, 2017,27(2):277295.
［11］Joachims T. Text categorization with support vector machines: Learning with many relevant features［C］// Proceedings of the 10th European Conference on Machine Learning. 1998:137142.
［12］Herrera F, HerreraViedma E, Martinez L. A fuzzy linguistic methodology to deal with unbalanced linguistic term sets［J］. IEEE Transactions on Fuzzy Systems, 2008,16(2):354370. 
［13］刘冬瑞,刘东升,张丽萍,等. 基于贝叶斯网络预测克隆代码质量［J］. 计算机科学, 2017,44(4):165168.
［14］Schumann S, Bühligen U, Neumuth T. Distance measures for surgical process models［J］. Methods of Information in Medicine, 2013,52(5):422431.
［15］Tang Jie, Yao Limin, Zhang Duo, et al. A combination approach to Web user profiling［J］. ACM Transactions on Knowledge Discovery from Data, 2010,5(1): Article No. 2.

[1]	WANG Hong-jie, XU Sheng-chao. Clustering Method of Cloud Platform Abnormal Transmission Data Based on Hilbert Similarity [J]. Computer and Modernization, 2023, 0(09): 27-31.
[2]	WANG Hong, GE Hong. Cross Modal Hash Retrieval Based on Attention Mechanism and Semantic Similarity [J]. Computer and Modernization, 2023, 0(08): 44-53.
[3]	LIU Guo-li, XU Hong-nan, TAN You-qian. Collaborative Filtering Recommendation Algorithm Combined with Expert Trust [J]. Computer and Modernization, 2022, 0(11): 60-68.
[4]	TIAN Feng, DENG Xiao-ping, ZHANG Gui-qing, WANG Bao-yi. A Non-intrusive Load Monitoring Method Based on Improved kNN Algorithm and Transient Steady State Features [J]. Computer and Modernization, 2022, 0(10): 29-35.
[5]	ZENG Yi-bin, GE Hong. Cross-modal Retrieval Based on Context Fusion and Multi-similarity Learning [J]. Computer and Modernization, 2022, 0(08): 50-56.
[6]	WANG Jun, HE Jin-rong, MA Le-rong. Character Network Analysis of Ordinary World [J]. Computer and Modernization, 2022, 0(06): 32-36.
[7]	ZOU Meng-yuan, FAN Zhi-qiang, XU Luo, LIU Jie, LIANG Wan-lu. Similarity Measurement Method of Inf-ProA Information Activity Process Model [J]. Computer and Modernization, 2022, 0(02): 26-32.
[8]	ZHU Ding-kai, TIE Zhi-xin, HONG Shun-he. An Initialization Algorithm of HRG Model and Its Application in Link Prediction [J]. Computer and Modernization, 2022, 0(02): 38-44.
[9]	WU Yang１, WU Guo-wen１, ZHANG Hong１, SHEN Shi-gen２, CAO Qi-ying. Rumor Source Detection Based on Extended Epidemic Model [J]. Computer and Modernization, 2022, 0(01): 113-119.
[10]	GENG Hua-cong, LIANG Hong-tao, LIU Guo-zhu. Recipe Recommendation Algorithm Based on Knowledge Graph and Collaborative Filtering [J]. Computer and Modernization, 2021, 0(08): 24-29.
[11]	WAN Yang-ye, GUO Jin-li. Link Prediction Algorithm Based on Resource Allocation and Graph Embedding Weighting [J]. Computer and Modernization, 2021, 0(07): 12-17.
[12]	HUANG Hao, CHEN Li. A Collaborative Filtering Algorithm Based on Information Entropy and Improved Similarity [J]. Computer and Modernization, 2021, 0(06): 29-34.
[13]	SHU Peng, DU Qing-wei. A Method for Mobile Community Detection Based on Multi-dimensional Informational Fusion [J]. Computer and Modernization, 2021, 0(05): 88-92.
[14]	SHAO Meng-qiao, JI Shun-hui, ZHANG Peng-cheng. AC-Rec: Academic Collaborators Recommendation Method Based on Multi-features [J]. Computer and Modernization, 2021, 0(03): 94-100.
[15]	YANG Quan. Semantic Similarity Calculation of Cilin Based on Logistic Function [J]. Computer and Modernization, 2021, 0(01): 111-119.