Explainable machine learning audit of attribute rating relationships in Football Manager 2023
Year: 2026
Type: article
Abstract: BACKGROUND: Football Manager data contain simulated player attributes generated within a proprietary game environment. They can reveal relationships within the rating system but cannot establish real-world talent-identification validity. OBJECTIVE: To audit how baseline attributes predict an internal Football Manager 2023 (FM2023) rating threshold, compare four untuned models under repeated cross-validation, and examine model-specific attribute-contribution stability across categories and positions. METHODS: We analyzed 2,282 anonymized male player profiles aged 15-18 years. The outcome was an FM2023 overall score of at least 10. Logistic regression, random forest, radial support vector machine (SVM), and a single-hidden-layer neural network were evaluated with three repeats of stratified five-fold cross-validation. We report receiver operating characteristic area under the curve (ROC-AUC), precision-recall AUC (PR-AUC; average precision), F1 score, sensitivity, specificity, balanced accuracy, Matthews correlation coefficient (MCC), and Brier score. Logistic-regression contributions were calculated on the logit scale; position-specific top-five stability was evaluated with 1,000 bootstrap samples per modeled position. RESULTS: Logistic regression had the highest mean ROC-AUC (0.779 ± 0.033) and PR-AUC (0.380 ± 0.047), whereas the neural network had the highest mean F1 score (0.284 ± 0.055). At the prespecified 0.50 threshold, logistic-regression sensitivity was 0.132 and specificity was 0.983. Technical, physical, and mental attributes appeared among the ten highest-ranked contributions. Mean absolute contribution per feature was slightly lower for technical than physical attributes (difference -0.0043). Position-specific rankings differed, but several top-five features had bootstrap inclusion frequencies below 50%. CONCLUSIONS: Baseline simulated attributes showed moderate discrimination for an internal FM2023 rating threshold, but sensitivity was low and position rankings were not uniformly stable. The results describe the internal structure of this simulation dataset and do not validate real-world coaching, scouting, or player-selection decisions.
Source: PLoS ONE
Authors: Zijian Zhang(Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing), Zhuyue Zhang(Data curation, Investigation, Validation, Visualization, Writing – review & editing), Lin Qiu(Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Software, Supervision, Writing – original draft, Writing – review & editing)
Institution: Soochow University
Language: English
Subfield: Economics and Econometrics
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,382
Award: G9001395956
OpenAlex ID: W7234914004
