A Simple Load Rebalancing algorithm to rebalance the loads in clouds

Distributed file systems are key building blocks for cloud computing applications based on the MapReduce programming paradigm. In such file systems, the nodes are simultaneously serve this computing and storage functions; a file is partitioned into a number of chunks allocated in distinct nodes so that this MapReduce tasks can be performed in parallel over the nodes. However, in a cloud computing environment, failure is the norm, and nodes may be upgraded, replaced, and added in the system. Files can also be dynamically created, deleted, and appended. This results in load imbalance in a distributed file system; that is, the file chunks are not distributed as uniformly as possible among the nodes. Emerging distributed file systems in production systems strongly depend on a central node for chunk reallocation. In this paper, a fully distributed load rebalancing algorithm is presented to solve with the load imbalance problem. This algorithm is compared against a centralized approach in a production system and a competing distributed solution presented in the literature. The simulation results indicate that our proposal is comparable with the existing centralized approach and considerably outperforms the prior distributed algorithm in terms of load imbalance factor, movement cost, and algorithmic overhead.