计算机与现代化
• 应用与开发 • 上一篇 下一篇
收稿日期:
出版日期:
发布日期:
作者简介:
基金资助:
Received:
Online:
Published:
摘要： 针对水利普查数据海量、多维的特点，研究近年来在“大数据”概念下发展迅速的Hadoop与Hive，结合传统数据仓库在多维数据分析方面的成熟技术，提出基于Hive的水利普查数据仓库的构建方法，描述数据仓库系统的架构，并根据Hive的设计特点，通过分桶、消减维度表和冗余事实表的方法来改进传统的多维分析模型,最后搭建集群系统对水利普查数据集进行查询与分析测试。测试结果表明该数据仓库可以满足海量多维水利普查数据的存储与查询要求。
关键词: Hive, 数据仓库, 水利普查, 模型优化, 大规模数据处理
Abstract: For the characters that water census data is of large volumes and high dimension, studying Hadoop and Hive which have a quick development recently in the “big data” concept and combining mature technology in multidimensional data analysis using traditional data warehouse, this article proposes a construction method of water census data warehouse based on Hive. This paper describes the architecture of data warehouse system, improves multidimensional model by dimension table reduction, fact table redundancy and Hive’s bucket method, then carries on queries and analysis to water census data set on Hadoop cluster system. Experimental results show that the data warehouse meets the f storage and query requirements of massive multidimensional water census data.
Key words: Hive, data warehouse, water census, model optimization, large data processing
中图分类号:
TP311.13
陈 龙,万定生,顾昕辰. 基于Hive的水利普查数据仓库[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2014.05.028.
CHEN Long, WAN Ding-sheng, GU Xin-chen. Water Census Data Warehouse Based on Hive[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2014.05.028.
0 / / 推荐
导出引用管理器 EndNote|Ris|BibTeX
链接本文: http://www.c-a-m.org.cn/CN/10.3969/j.issn.1006-2475.2014.05.028
http://www.c-a-m.org.cn/CN/Y2014/V0/I5/127
