计算机与现代化

• 算法设计与分析 • 上一篇    下一篇

基于超链接和DOM结构树的网页标题实时抽取方法

  

  1. 1.安徽大学计算机科学与技术学院,安徽合肥230601;
    2.安徽省工业图像处理与分析重点实验室,安徽合肥230039
  • 收稿日期:2015-03-16 出版日期:2015-08-08 发布日期:2015-08-19
  • 作者简介:张兵(1989-),男,安徽灵璧人,安徽大学计算机科学与技术学院硕士研究生,研究方向:网络信息处理与模式识别; 汤进(1976-),男,教授,博士生导师,博士,研究方向:模式识别; 罗斌(1963-),男,教授,博士生导师,博士,研究方向:模式识别。
  • 基金资助:
    国家高技术研究发展计划(863)项目(2014AA015104); 国家自然科学基金资助项目(61472002); 国家科技支撑计划项目(2012BAH95F00); 安徽省电力公司科技项目(521200130M0U, 5212M01353B4); 安徽省科技计划项目科技攻关计划(科技强警)(1301b042002)

Web Page Title Real-time Extraction Method Based on Hyperlink and DOM Tree

  1. 1. School of Computer Science and Technology, Anhui University, Hefei 230601, China;
    2. Key Lab of Industrial Image Processing & Analysis of Anhui Province, Hefei 230039, China
  • Received:2015-03-16 Online:2015-08-08 Published:2015-08-19

摘要: 网页标题的正确抽取,在Web文本信息抽取领域有着重大意义。本文提出一种网页标题实时抽取方法。首先通过对目录型网页进行实时解析,接着采用基于超链接遍历的方法,并利用标题与发布时间的对应关系,最终获取对应目录型网页的URL及锚文本。若获得锚文本不是网页正文的标题,则获取主题型网页的HTML源码并构建网页DOM树。在此基础上,结合网页标题的视觉特点,深度优先遍历DOM树,正确提取网页正文标题。实验结果表明,本文提出的Web网页标题实时抽取方法,具有实现简单,准确率高等优点。

关键词: 超链接, 网页标题, 发布时间, 实时, 信息抽取

Abstract: Correct extraction of Web title is significant to Web text information mining. This paper proposed a method which can get a real-time Web page title extraction. This method first used a real-time analysis model though the catalog page, and then used the hyperlink-based travelsal approach, and used the correspondence between the title and the release time to get the URL of the page and the corresponding anchor text. If the anchor text we have was not the title of the text page, we should get the Web page HTML source code and build a DOM tree for the corresponding theme-based Web page. Based on the visual characteristics of the Web page title, we traversed the DOM tree in depth-first order. The experimental results demonstrate that this method is of high accuracy and can be simply implemented and so on.

Key words: hyperlink, Web page title, publication time, real-time, information extraction

中图分类号: