Computer and Modernization

Previous Articles     Next Articles

Water Census Data Warehouse Based on Hive

  

  1. College of Computer and Information, Hohai University, Nanjing 211100, China
  • Received:2014-02-25 Online:2014-05-28 Published:2014-05-30

Abstract: For the characters that water census data is of large volumes and high dimension, studying Hadoop and Hive which have a quick development recently in the “big data” concept and combining mature technology in multidimensional data analysis using traditional data warehouse, this article proposes a construction method of water census data warehouse based on Hive. This paper describes the architecture of data warehouse system, improves multidimensional model by dimension table reduction, fact table redundancy and Hive’s bucket method, then carries on queries and analysis to water census data set on Hadoop cluster system. Experimental results show that the data warehouse meets the f storage and query requirements of massive multidimensional water census data.

Key words: Hive, data warehouse, water census, model optimization, large data processing

CLC Number: