Work

T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects

Year: 2026
Type: article
Abstract: High-speed six-degree-of-freedom (6-DoF) visual tracking often requires a pre-existing computer-aided design (CAD) model, limiting its use with previously unseen objects. We present Trackable 3D Model Generation (T3DMG), a two-stage framework that uses red–green–blue (RGB) images to acquire a trackable model from two object views before online tracking. The acquisition stage combines foreground-mask refinement, zero-shot 3D generation, and mask-constrained cross-view point-cloud correction. The corrected model is then loaded into Fast-PWP3D for contour-and-texture tracking. On three objects with CAD references, corrected two-view reconstruction increases mean full-angle projection intersection over union from 77.59% to 81.67% relative to direct two-view fusion, and from 73.70% to 81.67% relative to the better single-view setting. Model acquisition takes approximately 100 s typically and 125 s in the reported worst case, before tracker-side model loading. The subsequent online Fast-PWP3D stage reaches up to 420 Hz, with successful rotation tracking up to 600 rpm in the tested setup. Experiments on three additional objects without CAD references demonstrate tracking feasibility. The results support limited-view model acquisition for high-speed visual sensing when pre-tracking acquisition time is acceptable. CAD models serve only as evaluation references.
Source: Sensors
Institution: Chiba University
Language: English
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7236750657
FWCI:
Cites: 33
Cited by:
Related to: 10