Work

When mining maps disagree: a controlled study of supervision choice and land-area error

Year: 2026
Type: preprint
Abstract: Public mining inventories can disagree about the extent of the same mine, which changes both the supervision a model receives and the score it is given. We compare alternative masks on identical 2019 EuroMineNet imagery, with fixed training and spatially grouped partitions of 50 training, 10 validation and 25 test complexes. In the original 36-fit experiment, agreement-only (consensus) supervision increased U-Net area MAE by 3.27 km² relative to Tang-only supervision and by 2.42 km² relative to class-count-matched random masking; these intervals condition on three fitted seeds. A twelve-fit FPN subset was inconclusive. Post-test diagnostics motivated a separately planned extension of 26 further runs, including one exact reproduction, five additional seeds for three policies and exact per-crop controls. With eight seeds, the original-cohort difference between consensus and random masking in MAE is close to zero. On a frozen external cohort of 78 complexes, neither primary penalty is resolved by intervals that resample both complexes and fitted seeds: the consensus MAE reduction is -19.45 km² against Tang (97.5% interval -73.84 to 37.84) and -6.09 km² against random masking (-63.96 to 56.72). External IoU is low for every policy, and variation between seeds is substantial. A secondary analysis of frozen three-seed models finds a small interaction between training inventory and scoring reference of 0.0062 IoU (95% interval 0.0022-0.0102). The external reference closely follows one of the source inventories, so it cannot serve as independent truth. The results establish neither a stable penalty nor a stable benefit from consensus supervision. In this mining benchmark, conclusions about supervision policy changed with the number of training seeds, the evaluation cohort and the reference used for scoring. Third-party material retains the licences identified in the manuscript.
Language: English
Open Access status: green
OpenAlex ID: W7236775000
Cites:
Cited by:
Related to: 10