Work

Label validity before model choice: a leakage-aware evaluation of early-warning prediction in enterprise event logs

Year: 2026
Type: article
Abstract: To examine which early-warning claims remain defensible when event-log features, duration thresholds, and development labels are restricted to information available at deployment. Three public logs (188,944 cases; 1,850,318 events) were split chronologically by case arrival into training, calibration, validation, and test sets. Kaplan–Meier duration thresholds used historical training events only; development prefixes and label-availability times preceded the first test arrival. Logistic regression and LightGBM were compared with a GRU benchmark at event-count checkpoints. Uncertainty used case, arrival-month, circular moving-block, and short-window weekly bootstrap schemes. Eleven of 32 in-scope configurations were evaluable. Loan delay AUC ranged 0.639–0.705, permit repetition AUC 0.530–0.932, and traffic-fine repetition AUC 0.485–0.723. Trajectories describe selected populations, not learning effects; 157 of 562 common permit cases changed labels. Concept additions produced small AUC changes, and the GRU showed no uniform advantage. Moving-block intervals condition on fixed fitted models and therefore do not include model-selection or fitting uncertainty in this final revision. The study provides an auditable reporting protocol that makes label availability, non-evaluable tasks, checkpoint selection, temporal dependence, and the limits of observational guidance explicit before model comparisons are interpreted.
Source: Figshare
Language: English
Open Access status: green
OpenAlex ID: W7236606974
FWCI:
Cites:
Cited by:
Related to: 10