Work

Bandwidth-Aware Routing Mechanism to Control Hadoop Shuffle Traffic over SoftwareDefined Networking

Year: 2025

Type: conference-paper

Abstract: MapReduce is the main program in the Hadoop computing architecture.It involves mapping and reduction processes that require servers to exchange large amounts of data with each other.Data exchange between servers during the shuffle stage can lead to insufficient network bandwidth.In the present study, an algorithm for distributing Hadoop shuffle traffic to all possible paths was developed.Software-defined networking (SDN), a centrally controlled form of network architecture, was used to collect network status parameters to allocate shuffle traffic.During traffic distribution, the Mininet simulator was used to build the network topology and simulate operation.The Ryu controller was used as the SDN controller.The simulation results indicated that the proposed method was superior to the Spanning Tree Protocol (STP) and bandwidth-aware algorithm in reducing Hadoop completion time through effective shuffle traffic distribution.

Source: Proceedings of International Conference on Artificial Life and Robotics

DOI: 10.5954/icarob.2025.os5-2

Authors: Ming-Syuan Wu, Cheng-Han Lin, Wen-Shyang Hwang, Ce-Kuen Shieh, Mao-Syun Lin

Institutions: National Kaohsiung University of Science and Technology, Fooyin University, National Cheng Kung University

Language: English

Topic: Software-Defined Networks and 5G

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funder: National Science and Technology Council

OpenAlex ID: W4409913858