Work

A Failure-Order Resilience Signature for Self-Healing AI Inference Services: A Renewal-Based Framework

Year: 2026
Type: article
Abstract: AI inference services used in production environments must not only produce highly accurate predictions but also continue providing service in the presence of hardware, software, network, and orchestration failures. Traditional reliability measures mostly focus on time-based indicators but do not directly show how many failure events a self-healing service can tolerate before its first complete service interruption. In this study, the failure-order resilience signature (FORS) is defined to measure the event-based resilience of a self-healing AI inference service running on two active replicas. In the homogeneous model, replica times to failure are assumed to be exponentially distributed, recovery times are assumed to follow a general distribution, and the probability of a successful recovery cycle is expressed through the Laplace–Stieltjes transform of the recovery-time distribution. It is shown that FORS has a shifted geometric distribution under exponential recovery, and extensions are developed for heterogeneous replicas, survivor overload, time-based reliability, and recovery-time variability. Monte Carlo simulations based on one million service lifetimes per scenario validated the analytical results, with relative errors of less than approximately 0.10% in all scenarios. The mean times to first outage for systems with the same FORS value but different absolute rate scales were 50, 25, and 12.5 time units. In the survivor overload scenario, the expected failure order at the first outage decreased to 3.476, and the mean time to first outage decreased to 19.708 time units. In the production-trace application using the Alibaba Cluster Trace GPU v2025 dataset, 23,871 inference instances belonging to 156 services were examined. Among 21 service layers satisfying the exact two-replica criterion, ten completed operational races ended in a 1→2 transition, whereas six ended in a 1→0 zero-instance transition. The operational cycle-success probability was estimated at 0.625, the expected failure order at the first outage at 3.667, and the probability of not experiencing a zero-instance outage over approximately 31 days at 0.682. Because the dataset does not label the causes of instance deletion, these empirical results are based on instance lifecycle transitions rather than confirmed failure events. The proposed FORS is not a distribution-free replacement for the classical system signature, but rather a complementary, recovery-aware measure of AI service resilience that accounts for the recovery mechanism and the number of failures tolerated before the first complete interruption.
Institution: Fırat University
Language: English
Open Access status: diamond
OpenAlex ID: W7233543273
FWCI:
Cites: 18
Cited by:
Related to: