计算机与现代化 ›› 2010, Vol. 1 ›› Issue (10): 23-28.doi: 10.3969/j.issn.1006-2475.2010.10.007
• 人工智能 • 上一篇 下一篇
王 凯，吴泉源，杨树强
收稿日期:
修回日期:
出版日期:
发布日期:
WANG Kai, WU Quan-yuan, YANG Shu-qiang
Received:
Revised:
Online:
Published:
摘要： 随着更多的企业开始使用数据密集型集群计算系统如Hadoop和Dryad实现了更多的应用，多用户间共享MapReduce集群这种既减少了建立独立集群的代价，同时又使得多用户间可以共享更多的大数据集资源的需求日益增多。在公平调度算法的基础上，结合槽分配延迟和优先级的技术，本文提出了一种改进算法，可以实现更好的数据本地性，改善整个系统的计算性能如吞吐率、响应时间等；同时为了满足差别化的商业服务，通过对用户设置相应的优先级保证紧急任务的完成。
关键词: 公平调度, 等待调度, MapReduce, Hadoop
Abstract: As more enterprises start to use data-intensive cluster computing systems such as Hadoop and Dryad for more applications, sharing MapReduce clusters among multiple users that reducing the cost of establishing an independent cluster and the demand of sharing common data sets resources for users is increasing. Based on fair scheduling algorithm, combining with slot allocation delay and priority technology, the paper proposes an improved algorithm. It can achieve better data locality, improve the performance of the system, such as throughput, response time. To meet the differentiated business services, it sets the appropriate for users to ensure special tasks.
Key words: fair schedule, wait schedule, MapReduce, Hadoop
中图分类号:
TP311
王凯;吴泉源;杨树强. 一种多用户MapReduce集群的作业调度算法的设计与实现[J]. 计算机与现代化, 2010, 1(10): 23-28.
WANG Kai;WU Quan-yuan;YANG Shu-qiang. Design and Implementation of Job Scheduling Algorithm for Multi-User MapReduce Clusters[J]. Computer and Modernization, 2010, 1(10): 23-28.
0 / / 推荐
导出引用管理器 EndNote|Ris|BibTeX
链接本文: http://www.c-a-m.org.cn/CN/10.3969/j.issn.1006-2475.2010.10.007
http://www.c-a-m.org.cn/CN/Y2010/V1/I10/23
