MDFlaker: A tool for multi-factor detection and root cause analysis of flaky tests
Year: 2025
Type: article
Abstract: Test flakiness, characterized by inconsistent test outcomes without changes to the codebase, is a prevalent issue in continuous integration (CI) environments. This paper introduces a practical tool named MDFlaker that implements a multi-factor approach for detecting flaky tests, grounded in empirical research. The tool analyzes four key factors—traceback coverage, test smells, flakiness frequency, and test size—and applies a K-Nearest Neighbors (KNN) model for classification. Depending on the test case’s execution history, the tool switches between traceback-prioritized or multi-factor classification strategy. Integrated with platforms like GitHub and Travis CI, it processes real-time data from commits, builds, and test executions to support both research experimentation and industrial deployment. By improving the accuracy and interpretability of flaky test detection, the tool helps developers regain trust in CI pipelines, reduces debugging overhead, and fosters high-quality, rapid software delivery. This work not only enhances software reliability in practice but also opens new avenues for research in automated software quality assurance. • A reusable tool (MDFlaker) for flaky test detection and root cause analysis. • Combines test smells, traceback coverage, flakiness frequency, and test size. • Uses conditional KNN classification based on test execution history. • Integrated with GitHub and Travis CI for real-time CI/CD feedback. • Enhances developer trust by reducing non-deterministic test failures.
Source: Software Impacts
Author: Azeem Ahmad
Institution: Prince Sultan University
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $600
Funder: Prince Sultan University
OpenAlex ID: W4416024585