水文大数据共享平台研究与设计
Research and Design of Hydrological Big Data Sharing Platform
DOI: 10.12677/JWRR.2018.71002, PDF,  被引量    国家自然科学基金支持
作者: 陈 华, 徐 坚, 陈 杰, 郭生练, 许崇育：武汉大学水资源与水电工程科学国家重点实验室，湖北 武汉；肖志远：长江水利委员会水文局，湖北 武汉；杨家伟：武汉大学水资源与水电工程科学国家重点实验室，湖北 武汉；水资源安全保障湖北省协同创新中心，湖北 武汉
关键词: 大数据；水文数据共享；分布式文件系统；分布式数据库；Big Data； Hydrological Data Sharing； HDFS； HBase
摘要: 本文根据水文数据的特点探讨了水文大数据标准化方法，探索数据预处理、数据索引、数据高效存储等水文大数据共享平台关键技术；利用Hadoop对多源异构数据的海量存储能力及高速计算能力，研究基于MapReduce的水文大数据分布式数据处理模型，设计和实现水文大数据共享平台，为水利及跨行业跨部门的信息共享、空间集成，以及跨学科的可持续发展研究提供技术支撑。
Abstract: In this paper, the methodology of hydrological big data standardization is discussed upon analyzing on the characteristics of hydrological data. Solutions on data preprocessing, data indexing and highly effi-cient data reading and writing are also introduced. The mass storage capacity and high speed computing capability of Hadoop are utilized for designing and implementing hydrological big data sharing platform. Accordingly, the platform can be technical support for information sharing and space integration between water conservancy industry and other industries, as well as the interdisciplinary sustainable development.
文章引用：陈华, 徐坚, 肖志远, 杨家伟, 陈杰, 郭生练, 许崇育. 水文大数据共享平台研究与设计[J]. 水资源研究, 2018, 7(1): 10-18. https://doi.org/10.12677/JWRR.2018.71002

参考文献

[1] 蔡佳男, 耿庆斋. 水利科学数据共享汇交体系探索与构建[J]. 中国水利水电科学研究院学报, 2006, 4(1): 31-35. CAI Jianan, GENG Qingzhai. Study on concept and data collection system of scientific data sharing of water resources. Journal of China Institute of Water Resources and Hydropower Research, 2006, 4(1): 31-35. (in Chinese)
[2] 郭亚曦. 我国气象科学数据共享系统建设与服务[J]. 中国科技资源导刊, 2008, 40(2): 14-18. GUO Yaxi. Meteorological science data sharing system construction and service in China. China Science & Technology Resources Review, 2008, 40(2): 14-18. (in Chinese)
[3] 陈军飞, 邓梦华, 王慧敏. 水利大数据研究综述[J]. 水科学进展, 2017, 28(4): 622-631. CHEN Junfei, DENG Menghua and WANG Huimin. A review of water resources big data. Advances in Water Science, 2017, 28(4): 622-631. (in Chinese)
[4] 莫荣强, 艾萍, 吴礼福, 等. 一种支持大数据的水利数据中心基础框架[J]. 水利信息化, 2013(3): 16-20. MO Rongqiang, AI Ping, Wu Lifu, et al. A fundamental frame of water resources data center supporting big data. Water Resources Informatization, 2013(3): 16-20. (in Chinese)
[5] AI, P., YUE, Z. X. A framework for processing water resources big data and application. Applied Mechanics and Materials, 2014: 3-8.
[6] 冯吉平, 陈微, 官涤, 等. 大数据技术在松辽流域水环境管理中的应用展望[J]. 水利发展研究, 2014, 14(9): 63-65. FENG Jiping, CHEN Wei, GUAN Di, et al. Application prospect of big data technology in water environment management of Songliao River Basin. Water Resources Development Research, 2014, 14(9): 63-65. (in Chinese)
[7] CHALH, R., BAKKOURY, Z., OUAZAR, D., et al. Big data open platform for water resources management. 2015 International Conference on Cloud Technologies and Applications (CloudTech), 2015: 1-8.
[8] LI, D., GUO, S. and YIN, J. Big data analysis based on POT method for design flood prediction. 2016 IEEE International Conference on Big Data Analysis (ICBDA), 2016: 1-5.
[9] 孙欣欣. 城市突发水涝灾害大数据分析技术研究[J]. 科技通报, 2016, 32(4): 196-201. SUN Xinxin. Study on the big data technology of the city unexpected flood disaster. Bulletin of Science and Technology, 2016, 32(4): 196-201. (in Chinese)
[10] 中华人民共和国水利部. SL323-2011实时雨水情数据库表结构与标识符[S], 2011. The Ministry of Water Resources of the People’s Republic of China. Structure and identifier for real-time hydrological information database, 2011. (in Chinese)
[11] 中华人民共和国水利部. SL324-2013基础水文数据库表结构与标识符标准[S], 2013. The Ministry of Water Resources of the People’s Republic of China. Standard for structure and identifier in fundamental hydrological database, 2013. (in Chinese)
[12] 中华人民共和国水利部. SL325-2014水质数据库表结构及标识符[S], 2014. The Ministry of Water Resources of the People’s Republic of China. Structure and identifier for water quality database, 2014.
[13] 李洁. 我国水利信息化建设现状及趋势[J]. 城市建设理论研究: 电子版, 2013(18): 2095-2104. LI Jie. The current situation and tendency of the construction of Chinese water conservation informatization. Urban Construction Theory Research, 2013(18): 2095-2104. (in Chinese)
[14] 贾仰文, 王浩, 彭辉. 水文学及水资源学科发展动态[J]. 中国水利水电科学研究院学报, 2009, 7(2): 81-88. JIA Yangwen, WANG Hao and PENG Hui. A review on international development trend in the field of hydrology and water resources. Journal of China Institute of Water Resources and Hydropower Research, 2009, 7(2): 81-88. (in Chi-nese)
[15] PROVOST, F., FAWCETT, T. Data science and its relationship to big data and data-driven decision making. Big Data, 2013, 1(1): 51-59. [Google Scholar] [CrossRef] [PubMed]