计算机与现代化

• 数据库与数据挖掘 • 上一篇    下一篇

基于业务分割的并行式大数据迁移策略研究

  

  1. 扬州大学信息工程学院,江苏扬州225127
  • 收稿日期:2014-08-06 出版日期:2014-11-27 发布日期:2014-12-10
  • 作者简介: 刘晴和(1993-),女,江苏扬州人,扬州大学信息工程学院本科生,研究方向:大数据处理; 杨云(1957-),男,江苏扬州人,教授,博士,研究方向:TCP/IP协议分析,无线网络技术; 贺兴亚(1986-),女,江苏泰州人,副教授,硕士,研究方向:计算机网络,并行处理技术。
  • 基金资助:
     国家自然科学基金资助项目(61003180/F020509); 江苏省自然科学基金资助项目(BK2010683)

 Research on Parallel Big Data Migration Strategy Based on Job Segmentation

  1. College of Information Engineering, Yangzhou University, Yangzhou 225127, China
  • Received:2014-08-06 Online:2014-11-27 Published:2014-12-10

摘要:  大数据处理是目前研究的一个热点问题,大数据给数据存储、数据管理、数据检索带来巨大的挑战,它对存储硬件、存储策略、检索方法等研究提出了更高的要求。针对大数据处理问题,提出基于业务分割的、并行式数据迁移策略,并在此基础上开发数据迁移平台。实验结果表明:该平台在数据访问速度、占用系统内存等方面,比传统的数据迁移方法拥有更大的优势。目前该平台已在某银行收支核查系统中取得了很好的使用效果。

关键词:  , 大数据处理, 存储策略, 数据迁移, 业务分割, 并行计算

Abstract: Theres no doubt that Big Data has become a research focus recently. Not only because Big Data has posed great challenges to issues such as data storage, management and retrieval, but due to the fact that it draws higher demand in the study of storage hardware, storage strategy and retrieval method. In view of the processing problem of Big Data, the study of parallel data migration strategy based on job segmentation has been put forward. Moreover, a data migration platform has been developed under the guidance of the study. Just as results of the experiment show, this platform not only gains an edge in data access speed and system memory compared with traditional data migration methods, but also has achieved desirable effects in the application of a bank balance verification system.

Key words: Big Data, storage strategy, data migration, job segmentation, parallel computing