Mining Two Decades of Repository History to Predict Software Complexity Accumulation: A Leakage-Aware Landmark Survival Analysis
Year: 2026
Type: preprint
Abstract: Context: Software repositories accumulate decades of fine-grained change history, yet the analytic value of that history remains underexploited: complexity metrics are widely used to flag risky code, but nearly all empirical evaluations rely on static snapshots, so it remains unknown whether early measurements mined from a file's first commits predict how much complexity it will subsequently accumulate. Objective Complexity accumulation is modelled as a time-to-event process to determine (RQ1) which signals observable in a file's earliest commits predict how soon its complexity grows substantially beyond its initial level, (RQ2) whether richer structural information improves such forecasts beyond early process dynamics, and (RQ3) whether the resulting model generalises to projects unseen during training. Method An automated mining pipeline traversed fourteen Python open-source systems in chronological order and extracted 36,076 file-level commit observations spanning 2005–2026, yielding 1,291 per-file trajectories. A relative growth event, defined as a file's maximum cyclomatic complexity rising δ units above its own initial value, is combined with a landmark design in which every predictor is measured only within a file's first three commits. Together these eliminate event tautology and outcome leakage by construction, and two automated guards halt the pipeline if either property is violated. Penalized Cox proportional-hazards models were then evaluated by repeated cross-validation with paired confidence intervals against reimplemented size, static-complexity, coupling, and process baselines. Results A concordance of 0.663 (95% CI 0.626–0.704) was achieved, which is + 0.086 (+ 0.023 to + 0.131) higher than the best baseline and, in eight configurations out of nine, was significantly higher. Initial static complexity barely exceeded chance (0.541). Neither structural covariates (+ 0.014) nor gradient boosting (+ 0.021) produced supported gains, and leave-one-project-out concordance fell to 0.599. Conclusion Complexity accumulation is governed by early development dynamics rather than by initial code structure. Within-corpus forecasts are reliable, whereas cross-project transfer is weak and calls for per-organisation calibration. A fully automated, guard-checked mining and analysis pipeline is released to support replication.
Source: Research Square
Author: Moatasem M. Draz
Institution: Kafrelsheikh University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7226019240
