Year: 2024
Type: dissertation
Abstract: Distributed systems are gaining widespread adoption across IT organizations. Monitoring failures in these systems, which consist of loosely coupled applications, is cumbersome and requires manual attention. This study focuses on implementing anomaly detection within a sandbox running Kafka to detect failures automatically. To train and test the models, ”Chaos Engineering” is used to inject controlled failures into the system. Since the sandbox is not currently under load, a load simulator is created to mimic five different scenarios: constant load, linear increase load, linear decrease load, sine load, and a real-life scenario load. The study also investigates the ability to forecast metrics over various forecasting horizons, from 5, 10, to 30 minutes into the future. The forecasting model showed varying performance results depending on the current load on the sandbox and the metric being forecasted, as some metrics showed high volatility, leading to worse forecasting performance. Overall, increasing the forecasting horizon led to worse predictions, but within reasonable margins. The study concluded that the CPU usage metric performed best for the real-life simulation as well as for the sine load across all models. For the linear increase, the metric for consumer group lag was the best for all models. This metric also performed best for LSTM during the linear decrease load; however, KNN’s best metrics were network error increase and memory usage. The best metric for Isolation Forest was topic offset. For the constant simulation, Kafka latency was the best metric for LSTM and KNN, while network error performed best for Isolation Forest. In terms of overall model performance, KNN was the best for the real-life simulation and the linear increase load. Isolation Forest performed best for the sine, linear decrease, and constant simulations. The study also concludes that Chaos Engineering within the sandbox was able to inject enough errors for the models to react differently to the loads compared to the regular load simulations. The novel approach introduced in this study offers a methodology for building a machine learning model within a test environment using Chaos Engineering without the need for production data or real-life usage.
Authors: Mircoram, Omid, Härdling, Emil
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7015285180