Abstract: In a GPU cluster, the static task scheduling policy may result in unbalanced allocation of computing resource, because GPUs work as co-processors. A weight-based dynamic scheduling policy is proposed and implemented as a middleware, so that it can be applied to the GPU cluster. Under this policy, local GPUs and remote GPUs are not explicitly distinguished, and no global information is required. Every cluster node decides whether to use local GPUs or remote GPUs, according to weights of GPUs. And these weights are locally maintained by each node, respectively. As a carrier of the policy, the middleware ensures that the policy is transparent to users. It is composed of three parts: API library, resource management daemon, and GPU execution daemon. The policy is validated on a two-node cluster. Experiments show that the weight-based dynamic scheduling policy can achieve a 16% higher GPU utilization rate than the static policy, and a 45% higher GPU utilization rate than another dynamic policy (global-queue-based policy).

Key words: GPU cluster, dynamic task scheduling, middleware