Cloud Data Lake Architecture for Real-Time 5G KPI Telemetry and Predictive Analytics
Year: 2026
Type: conference-paper
Abstract: The 5G Standalone (SA) networks have brought about a paradigm shift in the telecommunications industry and is marked by the outburst of high-velocity telemetry data to support Ultra-Reliable Low-Latency Communications (URLLC). Conventional centralized monitoring systems that can be based on relational database management systems (RDBMS) and batch-oriented ETL operations are not able to support the sub-milliseconds latency of modern network slicing and on-demand resource allocation. This paper seeks to discuss these limitations by suggesting a scalable architectural design of a cloud-native Data Lake that implements a scalable layering approach known as a Medallion to separate storage and compute. The suggested system, in which Apache Kafka is combined with Apache Flink, will convert raw, semi-structured telemetry data on gNodeB and 5G Core functions into actionable intelligence, bypassing the bottlenecks that came with the legacy infrastructure. Alongside real-time monitoring, this study authenticates a new predictive analytics engine that would enable the active operations of the network. We use a hybrid machine learning framework which is a combination of XGBoost and Gated Recurrent Units (GRU) to predict some important Key Performance Indicators (KPIs), including throughput and cell congestion. There are experimental outcomes of a high-fidelity Digital Twin simulation that prove that this architecture decreases the data processing latency more than 90 times in contrast to the old systems, with final end-to-end visibility in less than 800 milliseconds. In addition, the hybrid predictive model attained an $R^2$ value of$\mathbf{0.985}$, which demonstrates its effectiveness due to its ability to model the non-linear and bursty nature of traffic in 5 G surroundings. This data validates the necessity of a decoupled, cloud-native architecture, which is necessary to facilitate the next generation of self-sustaining and automated networks.
Author: Saurabh Hitendra Patel
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7167486147