T3DMG: RGB-Based Trackable 3D Model Generation for High-Speed 6-DoF Tracking of Previously Unseen Objects
Year: 2026
Type: article
Abstract: High-speed six-degree-of-freedom (6-DoF) visual tracking often requires a pre-existing computer-aided design (CAD) model, limiting its use with previously unseen objects. We present Trackable 3D Model Generation (T3DMG), a two-stage framework that uses red–green–blue (RGB) images to acquire a trackable model from two object views before online tracking. The acquisition stage combines foreground-mask refinement, zero-shot 3D generation, and mask-constrained cross-view point-cloud correction. The corrected model is then loaded into Fast-PWP3D for contour-and-texture tracking. On three objects with CAD references, corrected two-view reconstruction increases mean full-angle projection intersection over union from 77.59% to 81.67% relative to direct two-view fusion, and from 73.70% to 81.67% relative to the better single-view setting. Model acquisition takes approximately 100 s typically and 125 s in the reported worst case, before tracker-side model loading. The subsequent online Fast-PWP3D stage reaches up to 420 Hz, with successful rotation tracking up to 600 rpm in the tested setup. Experiments on three additional objects without CAD references demonstrate tracking feasibility. The results support limited-view model acquisition for high-speed visual sensing when pre-tracking acquisition time is acceptable. CAD models serve only as evaluation references.
Source: Sensors
DOI: 10.3390/s26227324
Authors: Zejing Zhao(Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing), Cheng Ju(Conceptualization, Investigation, Methodology, Validation), Daigo Fujiwara(Methodology, Supervision), Satoshi Suzuki(Methodology, Supervision), Akio Namiki(Investigation, Methodology, Supervision, Writing – review & editing)
Institution: Chiba University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Keyword: two-view reconstruction
Open Access status: gold
Estimated APC paid: $3,137
Award: G1804817109
OpenAlex ID: W7236750657
