基于DOM树统计关键词路径的WEB信息提取研究

Research on WEB Information Extraction Based on DOM Tree Statistics Keyword Path

摘要: 根据用户的需求提取WEB信息在WEB数据挖掘领域中起着重要的作用。本文以提取公司官网上的产品列表为目标，提出了一种基于DOM树统计关键词路径的方法来确定关键路径，根据关键路径来确定提取内容。过程主要分为候选关键词组的获取、企业公司官网产品页面的获取、web页面的DOM树建立以及确定关键路径及提取。本文提出了一种解决不同公司官网上网页结构差距较大问题的信息提取的方法。并且根据该方法实现了在汽车零部件生产厂商官网上提取需要的产品信息。

Abstract: Extracting WEB information according to users’ requirements plays an important role in WEB data mining. Aiming at extracting the list of products on the company website, this paper proposes a method based on DOM tree statistics keyword path to determine the critical path and extract content according to the critical path. It is mainly divided into the acquisition of key phrase candidates, the acquisition of the product page of the company’s official website, the establishment of the DOM tree of the web page, and the determination of the key path and extraction. This paper proposes an information extraction method to solve the problem of large difference in web page structure on different companies’ official websites. According to this method, the required product information is extracted from the official website of the auto parts manufacturer.