Authors: M. Shalima Sulthana, Mrs. K. Bhanu Sri, Ms. B. Ramya Sri
Abstract: Map Reduce is a scheme for processing and managing large scale data sets in a distributed cluster, which has been used for applications such as document clustering, generating search indexes, access log analysis, and numerous other forms of data analytic. In the existing system, a hash function is used to partition intermediate data among reduce tasks and most of the previous algorithms proposed concentrated on other parameters like data uploading, time reduction etc. None of them dealt with network traffic. Our proposed system consists of a decomposition-based distributed algorithm to deal with the large-scale optimization problem for large data application and an online algorithm is additionally designed to adjust data partition and aggregation in a dynamic manner. The cost of Network traffic under both offline and on-line cases is significantly reduced as demonstrated by the extensive stimulation results by the various proposals considered and used.
Keywords: Big Bata, Data Aggregation, Dynamic Decomposition-based Distributed K- means Algorithm, HC Algorithm, Traffic Minimization.
Full Issue
| View or download the full issue | 16-28 PDF |