HOME JOURNALS CONTACT

Information Technology Journal

Year: 2014 | Volume: 13 | Issue: 3 | Page No.: 501-507
DOI: 10.3923/itj.2014.501.507
A Job Scheduling Policy based on the Job-classification and Dynamic Replica Mechanism
Yan Qiang, Yue Li, Wei Wei, Bo Pei, Juanjuan Zhao and Hui Zhang

Abstract: In order to guarantee the stability of the system performance and the high Qos(quality of service) of users, a new method based on the HDFS (Hadoop Distributed File System) was proposed which including a job type classification method and a dynamic replica manage mechanism. The method uses the job type classification method to select the I/O intensive job, in order to achieve more accuracy of the classification taken the heterogeneity of the jobs into consideration. For the classified jobs, a dynamic replica manage mechanism was used to determine whether to increase or decrease the number of copies on the specific data node. For a test of a cluster with 1 namenode and 20 data nodes, the method has a high performance. The theoretical and experimental analyses in this paper prove that the proposed method has the ability to improve the performance of HDFS effectively.

Fulltext PDF Fulltext HTML

How to cite this article
Yan Qiang, Yue Li, Wei Wei, Bo Pei, Juanjuan Zhao and Hui Zhang, 2014. A Job Scheduling Policy based on the Job-classification and Dynamic Replica Mechanism. Information Technology Journal, 13: 501-507.

© Science Alert. All Rights Reserved