Research on Hadoop Job Scheduling Based on an Improved Genetic Algorithm

Abstract

Hadoop job scheduling is an important influence factor in the performance of the Hadoop platform. Due to its characteristics, genetic algorithm (GA) has natural parallel advantages in job scheduling. But the scheduling based on the traditional simple genetic algorithm (SGA), in the aspect of the average execution time of job, the convergence speed and the selection of the optimal solution, is lack of consideration, which cannot make full expression of its advantages. We propose an improved genetic algorithm, which adopts the reserved strategy of optimal solution to speed up the convergence speed. The experimental results indicate that the proposed algorithm has a certain improvement on the scheduling efficiency and the performance of platform.