基于关键词相似度的Web用户挖掘研究与实现

The Study and Implementation of Web User Mining System Based on the Similarity of Words

摘要: 在Web挖掘极度盛行的今天，收集大量网络数据已经不是问题，而如何在海量数据中抽取去噪后的有用数据成为要解决的关键问题。本文研究将网站用户的搜索关键词分析聚类，作为用户的兴趣、爱好标签，以供运营分析人员参考。文中根据世界知识或分类体系计算词语语义距离后转化为词语相似度的方法，将词语间距离依据词频、词权重等因子加工计算出关键词集合间相似度矩阵后，用欧式距离表示其关键字集的相似度；之后聚类算法利用现有R软件中开源算法包——基于隐马尔科夫模型的depmix算法包进行的用户聚类算法。最终用某搜索引擎用户的真实数据，经过数据去噪后所得实验数据进行聚类，并于前台展示聚类及用户周边相关结果。

Abstract: Nowadays, as web mining is extremely prevalent, it is easy to collect huge amounts of data but to figure out which materials is useful to analyze after de-noising is more important. This article discusses how to use the result of user’s searching keywords clustering as the label of the client for operational analysts to refer to. The similarity between isolated words is calculated by turning the word semantic distance based on world knowledge or classification system. Then the similarity between clients (keyword sets) is defined as the Euclidean distance of a similarity matrix constituted by the similarities between keyword sets which determined by word frequency and word weight. The “depmix” package which based on the Hidden Markov Model in “R” software is used as the clustering algorithm and the user clustering result is displayed at last using the real data of the users of a search engine.