Network-Aware gRPC Streaming for Edge-to-Cloud Time-Series Data Ingestion: A Multi-Objective Optimization Framework with Reinforcement Learning and Federated Intelligence
Year: 2025
Type: conference-paper
Abstract: The proliferation of Internet of Things (IoT) devices and edge computing has created unprecedented demands for efficient time-series data ingestion from edge environments to cloud-based observability platforms. While gRPC has emerged as a high-performance communication protocol, its static con- figuration approach fails to adapt to the dynamic and het- erogeneous network conditions characteristic of edge-to-cloud deployments. This paper presents NetStream, a novel network- aware optimization framework for gRPC streaming in distributed Cortex deployments. NetStream introduces five key innovations: (1) a hybrid machine learning-based network condition pre- diction model that combines LSTM networks, Random Forest algorithms, and Deep Q-Network reinforcement learning for adaptive parameter tuning, (2) an adaptive protocol configuration mechanism with federated learning capabilities that dynamically adjusts gRPC parameters based on predicted network conditions and collaborative intelligence from multiple edge deployments, (3) a hierarchical streaming strategy that optimizes data flow across multi-tier edge deployments with intelligent load bal- ancing, (4) a novel context-aware compression algorithm that adapts compression strategies based on data characteristics and network conditions, and (5) a distributed consensus mechanism for maintaining configuration consistency across federated edge environments. Our comprehensive evaluation using real-world IoT workloads, synthetic network traces, and production de- ployments demonstrates that NetStream achieves 47% reduc- tion in end-to-end latency, 35% improvement in throughput, 28% reduction in data loss, and 23% improvement in energy efficiency compared to static gRPC configurations. Additionally, our federated learning approach reduces model training time by 62% while improving prediction accuracy by 18% across heterogeneous edge deployments.
Author: Bhole, Manas
Language: English
Topic: IoT and Edge/Fog Computing
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7104444538
FWCI:
Cites:
Cited by:
Related to: