Allowing test execution to continue after failures: tool support and an industrial case study for fault-tolerant online model-based testing
Year: 2026
Type: article
Abstract: Context Model-Based Testing (MBT) can be executed in either offline or online mode. While offline MBT pre-generates test cases that can be independently executed and resumed, online MBT dynamically interacts with the system under test by generating test steps on-the-fly based on runtime feedback. This adaptive exploration makes online MBT particularly attractive in industrial contexts; however, it also introduces a structural fragility: when an unexpected failure occurs during execution, the testing session often terminates prematurely, preventing further exploration of the model. Objective The objective of this study is to improve the robustness and usability of online MBT by enabling execution to continue in the presence of localized assertion failures without masking genuine faults. Method We propose two lightweight execution-level fault-tolerance heuristics: Retry Failed Node and Blacklist Failed Node. The heuristics were implemented in the GraphWalker MBT tool by extending its execution and exception-handling mechanisms and evaluated using a large-scale industrial MBT suite at Testinium. Results The retry heuristic mitigates transient and flaky failures, while the blacklist heuristic enables continued traversal after persistent localized failures. Together, they significantly reduce premature termination of online MBT execution with minimal and bounded overhead. Conclusion Fault-tolerant execution semantics are essential for making online MBT practical in real-world testing. The proposed heuristics improve execution continuity and support broader industrial adoption of online MBT.
Institutions: Queen's University Belfast, Azerbaijan Technical University
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,830
OpenAlex ID: W7153720637