In Hadoop, the Hadoop Distributed File System (HDFS) gives a profoundly trustworthy static replication strategy for processing of data, which makes various applications to rely on Apache Hadoop. However, accessing rate of every file is unique, maintaining similar replication factor for each file results in weakening of execution. Considering this problem, research approach is been proposed of utilizing predictive examination for partial replicating the data’s progressively. So to attain the greater efficiency, Dynamic Data Partial Replication (DDPR) algorithm is implemented which helps to avoid excess memory consumption. In proposed, we have executed the partial replication technique i.e., the data is fragmented and stored in four Hadoop servers partially to achieve fault tolerance. The proposed method provides the user requested files even when there is problem in any servers by this partial replication technique. Before replication process, the files uploaded is checked for duplication to avoid repetition by implementing Secure Hash Algorithm-1 (SHA) that applies hash code of data to check ownership with code verification thus reducing the excess consumption of storage. Finally our method is compared with ARM (Adaptive Replication Management) and ERMS (Elastic Replication Management System) in terms of execution time for various file size. The implemented result demonstrates that our proposed methodology gives much efficient performance, redundancy and overhead is received.
Volume 11 | 07-Special Issue
Pages: 778-787