基于网络爬虫的数字城市研究与分析
Research and Analysis of Digital City Based on Web Crawler
DOI: 10.12677/CSA.2018.88129, PDF,    国家自然科学基金支持
作者: 饶加旺：江苏省测绘工程院，江苏 南京;中国科学院南京地理与湖泊研究所，江苏 南京；杨颜颜：金陵中学龙湖分校，江苏 南京；马荣华*：中国科学院南京地理与湖泊研究所，江苏 南京；杜臣昌：青岛市城市规划设计研究院，山东 青岛
关键词: 数字城市；网络爬虫；文本挖掘；R语言；Digital City； Web Crawler； Text Mining； R Language
摘要: 针对数字城市研究手段不足，尤其在收集大量研究文献的基础上对数字城市进行整体研究上的欠缺，本文基于R语言和Selenium框架设计了稳定、高效的爬虫程序，获取了中国知网2018年5月前收录的数字城市为主题的研究文献，并建立了数字城市文献数据库和数字城市自动分词模型。通过分析数字城市研究的时序性、空间分布特征和研究热点，揭示了数字城市研究的发展历程、现状、发展趋势和研究热点。结果表明本文设计的网络爬虫程序在数字城市研究文献的收集方面具有可行性和有效性。
Abstract: Aiming at the shortage of the research on digital city, especially in collecting a large number of research literature on the basis of overall study on digital city which were lacked, stable and efficient crawler was designed based on the R language and the Selenium framework. Research papers of digital city before May 2018 collected on the National Knowledge Infrastructure (CNKI) were obtained through the web crawler, then the literature of database and the model of automatic word segmentation in terms of digital city were built. By analyzing the chronological research of digital city, the spatial distribution characteristics, and research focus in the research literature, the development of digital city research, the present situation, development trend and research hotspot were revealed. Results showed that the web crawler had designed in this article performing feasibility and effectiveness on collection digital city research literature.
文章引用：饶加旺, 杨颜颜, 马荣华, 杜臣昌. 基于网络爬虫的数字城市研究与分析[J]. 计算机科学与应用, 2018, 8(8): 1172-1182. https://doi.org/10.12677/CSA.2018.88129

参考文献

[1] 李德仁, 邵振峰, 杨小敏. 从数字城市到智慧城市的理论与实践[J]. 地理空间信息, 2011, 9(6): 1-5.
[2] Munzert, S., Rubba, C., Meißner, P., et al. (2015) Automated Data Collection with R: A Practical Guide to Web Scraping and Text Mining. John Wiley & Sons, Hoboken.
[3] Khurana, D. and Kumar, S. (2012) Web Crawler: A Review. International Journal of Computer Science & Management Studies, 12, 31-36.
[4] Widenius, M. (2002) Mysql Reference Manual. O’Reilly & Associates, Inc., Sebastopol, California.
[5] 雷晓庆, 刘晓雁. 论文关键词特征的统计分析[J]. 图书情报工作, 1998, 32(5): 19-20.
[6] 奉国和, 郑伟. 国内中文自动分词技术研究综述[J]. 图书情报工作, 2011, 54(2): 41-45.
[7] 于重重, 操镭, 尹蔚彬, 等. 吕苏语口语标注语料的自动分词方法研究[J]. 计算机应用研究, 2017, 34(5): 1325-1328.
[8] Sun, J.Y. jieba中文分词[EB/OL].
https://github.com/fxsjy/jieba, 2016-04-20.
[9] 王瑞雷, 栾静, 潘晓花, 等. 一种改进的中文分词正向最大匹配算法[J]. 计算机应用与软件, 2011, 28(3): 195-197.
[10] 赵知纬, 钱龙华, 周国栋, 等. 一个面向信息抽取的中文跨文本指代语料库[J]. 中文信息学报, 2015, 29(1): 57-66.
[11] 国家测绘局地名研究所. 中国地名录[M]. 北京: 中国地图出版社, 1997.
[12] 民政部全国行政区划查询平台[EB/OL]. http://xzqh.mca.gov.cn/map
[13] Collobert, R. and Weston, J. (2008) A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. International Conference on Machine Learning, Helsinki, 5-9 July 2008, 160-167. [Google Scholar] [CrossRef] 
[14] 仵克让. 适用于城市数字移动通信的最佳码速[J]. 通信学报, 1983(4): 87-92.
[15] 方维慰. 中国建设数字城市的难点与对策[J]. 软科学, 2004, 18(3): 71-73.
[16] 李景宜, 石长伟, 傅志军, 等. 首届“数字地球”国际会议在北京召开[J]. 地球信息科学学报, 2000, 2(1): 5-5.
[17] Kwartler, T. (2017) Text Mining in Practice with R. John Wiley & Sons, Ltd., Chichester, UK. [Google Scholar] [CrossRef] 
[18] Halvey, M.J. and Keane, M.T. (2007) An Assessment of Tag Presentation Techniques. International Conference on World Wide Web (WWW 2007), Banff, Alberta, Canada, 8-12 May 2007, 1313-1314. [Google Scholar] [CrossRef] 
[19] Lang, D. and Chien, G. (2018) WordCloud2[EB/OL].
https://CRAN.R-project.org/package=wordcloud2