基于Hadoop的药品安全舆情的话题跟踪模型与算法

Topic Tracking Model and Algorithms for Drug Safety Public Opinion Based on Hadoop

摘要: 大数据时代如何从网络媒体发布的药品安全事件、药品安全监管及药品安全形势等医药新闻报道中跟踪药品安全舆情，是卫生部门和医药企业研判舆情的关键。本文利用八爪鱼采集器从某医药网获取2012年12月25日到2015年4月29日间9888条医药新闻，经数据清洗和人工识别选取了8类药品安全领域话题，共5667例实验数据。采用Hadoop平台和朴素贝叶斯分类算法实现药品安全话题跟踪。研究结果表明基于Hadoop平台的朴素贝叶斯分类算法的准确率较好、召回率较差、调和平均指标F1值为0.57时模型整体最佳。

Abstract: Track public opinion of drug safety and from online media news is beneficial to the health departments and pharmaceutical companies judging public opinion and making decisions quickly, accurately and efficiently. The medical news from December 25, 2012 to April 29, 2015 from a medical network is obtained by the Octopus collector. After data cleaning and manual selection, the experiments data are 5667 medical news including 8 categories of drug safety topics. The Hadoop platform and Naive Bayes classification algorithm are used to track drug safety topics. The research results show that the Naive Bayes classification algorithm based on Hadoop platform has better accuracy, poor recall rate and the best overall model when the F1 value is 0.57.