• Dr.D. Nagaraju
• B. Sandeep

The challenge of efficiently analyzing geo-distributed datasets has driven cloud providers to deploy data-parallel jobs across various geographically dispersed locations, such as data centers and edge clusters, which are connected via wide-area network (WAN) links. Current state-of-the-art geo-distributed data analytic approaches often fall short in fully leveraging the available network and computational resources. This limitation arises primarily because these methods require synchronization at bottleneck sites to complete data transmission and computation phases before progressing. Additionally, these approaches struggle to adapt to the dynamic nature of network bandwidth and the variability in job parallelism. To address these issues, we introduce a novel Simultaneous Data Transfer and Processing (SDTP) mechanism designed to enhance wide-area data analytics by integrating considerations for network bandwidth fluctuations and job parallelism. The SDTP allows a site to begin computation as soon as the necessary input data is available, thereby enabling concurrent execution of input data loading, map, shuffle, and reduce phases without waiting for other sites to finish their previous phases. We further refine the SDTP approach by incorporating more precise time estimation techniques and adapting the mechanism to handle dynamic conditions. Trace-driven simulations reveal that SDTP significantly reduces wide-area analytic job response times by 19% to 72% compared to existing methods, marking a substantial improvement in the efficiency of geo-distributed data analytics.

Distributed Data Analysis, Parallel Job Execution, Reducing Job Response Latency

"Accelerating Wide-Area Data Analytics with Simultaneous Data Transmission and Processing", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.11, Issue 6, page no.184-188, June-2024, Available :http://www.jetir.org/papers/JETIRGL06032.pdf

"Accelerating Wide-Area Data Analytics with Simultaneous Data Transmission and Processing", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.11, Issue 6, page no. pp184-188, June-2024, Available at : http://www.jetir.org/papers/JETIRGL06032.pdf