ReSORT: LLM-Guided Policy Selection for New-ID Assignment Under Occlusion in Multi-Object Tracking
Year: 2026
Type: article
Abstract: Objects that reappear after occlusion can receive new identities when data association fails, fragmenting trajectories. We present ReSORT, an identity recovery framework that evaluates competing links using motion history, numerical association scores, and selective language-model evidence. Camera-compensated trajectory states are propagated through five motion hypotheses to construct a persistent candidate graph. A numerical ranker scores candidate links, and a frozen Qwen3.5-4B model reviews selected ambiguous cases using image crops and scene context. Candidate-order consistency checks filter the model responses before their relative evidence enters a global one-to-one assignment. Inference does not access ground truth; the numerical ranker is trained with supervision, and the evaluated reassociation uses subsequent observations offline. On seven MOT17 development videos, the complete pipeline produces 243 identity switches, compared with 427 for native ByteTrack and 299 for the historical automatic ReSORT reference, with IDF1 of 85.96% and HOTA of 78.06%. The historical comparison includes changes beyond the final language-model component, whose contribution is assessed separately through matched ablations. Experiments across seven tracker families on MOT17 and MOT20 show tracker-dependent benefits and sensitivity to evidence weighting. The results support selective language-model verification within a constrained association system while identifying limitations in response consistency, computational cost, and development-set evaluation.
Source: Applied Sciences
DOI: 10.3390/app162211445
Institutions: Korea Maritime and Ocean University, Tech University of Korea
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7236037126
