Subscribe Now Subscribe Today
Science Alert
 
Blue
   
Curve Top
Information Technology Journal
  Year: 2014 | Volume: 13 | Issue: 3 | Page No.: 501-507
DOI: 10.3923/itj.2014.501.507
 
Facebook Twitter Digg Reddit Linkedin StumbleUpon E-mail

A Job Scheduling Policy based on the Job-classification and Dynamic Replica Mechanism

Yan Qiang, Yue Li, Wei Wei, Bo Pei, Juanjuan Zhao and Hui Zhang

Abstract:
In order to guarantee the stability of the system performance and the high Qos(quality of service) of users, a new method based on the HDFS (Hadoop Distributed File System) was proposed which including a job type classification method and a dynamic replica manage mechanism. The method uses the job type classification method to select the I/O intensive job, in order to achieve more accuracy of the classification taken the heterogeneity of the jobs into consideration. For the classified jobs, a dynamic replica manage mechanism was used to determine whether to increase or decrease the number of copies on the specific data node. For a test of a cluster with 1 namenode and 20 data nodes, the method has a high performance. The theoretical and experimental analyses in this paper prove that the proposed method has the ability to improve the performance of HDFS effectively.
PDF Fulltext XML References Citation Report Citation
How to cite this article:

Yan Qiang, Yue Li, Wei Wei, Bo Pei, Juanjuan Zhao and Hui Zhang, 2014. A Job Scheduling Policy based on the Job-classification and Dynamic Replica Mechanism. Information Technology Journal, 13: 501-507.

DOI: 10.3923/itj.2014.501.507

URL: https://scialert.net/abstract/?doi=itj.2014.501.507

COMMENT ON THIS PAPER
 
 
 

 

 
 
 
 
 
 
 
 
 

 
 
 
 
 

Curve Bottom