TextClusteringRetrievalBasedonLDAModel

doi:10.3969/j.issn.1006-2475.2018.06.002

Abstract

Abstract: Thetraditionalmethodofjudgingthesimilarityoftwodocumentsdoesnottakeintoaccountthesemanticrelationbehindthetexts,resultinginalargedifferencebetweentheresultsreturnedbytheretrievalsystemandtheusersqueryrequirements.ThispaperpresentsatextclusteringmethodbasedonLDAtopicmodel.Firstly,theapplicationprincipleofLDAtopicmodelisintroducedandthebasicmethodoftextminingisexpounded,andthentheLDAtopicmodelisconstructed.TheGibbssamplingmethodisusedtoderivetheprobabilitydistributionofthecharacteristicwords.Finally,thesetsoftestdataareclusteredwiththeK-means+〖KG-*3〗+methodchosenbytheoptimizationclustercenter.AndthedesignedLDA-GibbsmodeliscomparedwiththetraditionalTF-IDFmodel.Experimentalresultsshowthatthismodelcanimprovetheretrievaleffectofdataandhasgoodpromotionalvalue.

Key words: topicmodel, textclustering, latentDirichletallocation(LDA), clusterevaluation, informationretrieval(IR)

CLC Number:

TP391.1

LIXiao-ye,LIChun-sheng,LILong,ZHANGKe-jia. TextClusteringRetrievalBasedonLDAModel[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2018.06.002.

References ［17］

［1］	王鹏,高铖,陈晓美.基于LDA模型的文本聚类研究［J］.情报科学,2015,33(1):63-68.
［2］	杨平,王丹,赵文兵.微博网站中面向主题的权威信息搜索技术研究［J］.计算机科学与探索,2013，7（12）：1135-1145.
［3］	董婧灵.基于LDA模型的文本聚类研究［D］.武汉：华中师范大学,2012.
［4］	唐晓波,房小可.基于文本聚类与LDA相融合的微博主题检索模型研究［J］.情报理论与实践,2013,36(8):85-90.
［5］	CaoBuqing,LiuXiaoqing,LiuJianxun,etal.Domain-awareMashupserviceclusteringbasedonLDAtopicmodelfrommultipledatasources［J］.InformationandSoftwareTechnology,2017,90:40-54.
［6］	李湘东,张娇,袁满.基于LDA模型的科技期刊主题演化研究［J］.情报杂志,2014,33(7):115-121.
［7］	HajjemM,LatiriC.CombiningIRandLDAtopicmodelingforfilteringmicroblogs［J］.ProcediaComputerScience,2017,112:761-770.
［8］	焦潞林,彭岩,林云.面向网络舆情的文本知识发现算法对比研究［J］.山东大学学报(理学版),2014,49(9):62-68.
［9］	马军红.文本聚类算法初探［J］.电子世界,2012(6):71-72.
［10］	江浩,陈兴蜀,杜敏.基于主题聚簇评价的论坛热点话题挖掘［J］.计算机应用,2013,33(11):3071-3075.
［11］	ClyneB,CooperJA,HughesCM,etal.Aprocessevaluationofaclusterrandomisedtrialtoreducepotentiallyinappropriateprescribinginolderpeopleinprimarycare(OPTI-SCRIPTstudy)［J］.Trials,2016,17,doi:10.1186/s13063-016-1513-z.
［12］	王振振,何明,杜永萍.基于LDA主题模型的文本相似度计算［J］.计算机科学,2013,40(12):229-232.
［13］	孟雪井,孟祥兰,胡杨洋.基于文本挖掘和百度指数的投资者情绪指数研究［J］.宏观经济研究,2016(1):144-153.
［14］	ZhangYi,ZhangGuangquan,ChenHongshu,etal.Topicanalysisandforecastingforscience,technologyandinnovation:Methodologywithacasestudyfocusingonbigdataresearch［J］.TechnologicalForecastingandSocialChange,2016,105:179-191.
［15］	ReddyASS,BrikMG,KumarJS,etal.Structuralandelectricalpropertiesofzinctantalumborateglassceramic［J］.CeramicsInternational,2016,42(15):17269-17282.
［16］	王军.热门微博话题事件主题聚类分析［D］.合肥：安徽大学,2016.
［17］	陈晓美.网络评论观点知识发现研究［D］.长春：吉林大学,2014.

[1]	ZHENG Li-rui, XIAO Xiao-xia, ZOU Bei-ji, LIU Bin, ZHOU Zhan. Named Entity Recognition in Electronic Medical Record Based on BERT [J]. Computer and Modernization, 2024, 0(01): 87-91.
[2]	XU Hong-kui, LI Zhen-ye, GUO Wen-tao, ZHAO Jing-zheng, GUO Xu-bin. Real-time Detection of Arbitrary Shape Scene Text Based on Segmentation [J]. Computer and Modernization, 2023, 0(11): 95-100.
[3]	CUI Shao-guo, ZHANG Gang, WANG Ao-di. Deep Cross Network Recommendation Model Based on Attention Perception [J]. Computer and Modernization, 2023, 0(07): 54-60.
[4]	JIA Jian-zhong. Handwritten Signature Identification Based on Wavelet Transform and CPN Network [J]. Computer and Modernization, 2020, 0(07): 27-31.
[5]	WU Di， AI Zhong-liang， LIU Zhong-lin， LI Chang-bao . Knowledge Portrait Model for Research Project Documents [J]. Computer and Modernization, 2020, 0(02): 60-.
[6]	XIANG Wei. Reviews on Event Knowledge Graph Construction Techniques and Application [J]. Computer and Modernization, 2020, 0(01): 10-.
[7]	YUE Yi-feng, HUANG Wei, REN Xiang-hui. An Automatic Text Summarization Model Construction Method Based on BERT Embedding [J]. Computer and Modernization, 2020, 0(01): 63-.
[8]	LI Pan1, LI Yi-guang2, XU Chun1. Hot Network Information Extraction Based on Key Nodes [J]. Computer and Modernization, 2019, 0(09): 60-.
[9]	ZHANG Hao1,2, ZHONG Min1,2. Chinese Short Text Classification Based on Sentence-LDA Topic Model [J]. Computer and Modernization, 2019, 0(03): 102-.
[10]	FENG Jun, LIU Jing-hua, KONG Sheng-qiu. Chinese Collective Entity Linking Method Based on Multiple Features [J]. Computer and Modernization, 2019, 0(01): 69-.
[11]	SHEN Ran, LIN Kai-feng, WU Hui. A Multi-modal and Multi-task Framework for Power Supply Service Evaluation [J]. Computer and Modernization, 2018, 0(12): 51-.
[12]	SONG Ping1,2,3, XU Guang-luan2,3, ZHOU Yan-hai4, GUO Zhi2,3, YAN Meng-long2,3, ZHANG Yi-fei2. Aircraft Detection Method Based on MRNSSD Model for Remote Sensing Images [J]. Computer and Modernization, 2018, 0(12): 110-.
[13]	WANG Hai-peng, ZHENG Yang-fei. Lawyers Recommendation Algorithm and Improvement Scheme Based on Eigenvalue [J]. Computer and Modernization, 2018, 0(10): 18-.
[14]	LIANG Dong, YANG Yong-quan, WEI Zhi-qiang. Information Extraction of Web Pages Based on Support Vector Machine [J]. Computer and Modernization, 2018, 0(09): 21-.
[15]	WANG Liang-yu. Knowledge Graph Construction Method for Carbon Trading [J]. Computer and Modernization, 2018, 0(08): 114-.