Work

ビジュアルフィードバックシステムのための時間マッピングおよび時間予測に基づく超低遅延物体追跡コンピューティングアーキテクチャ

Year: 2023
Type: article
Abstract: Visual feedback system, which combines virtual (computing) and real spaces by sensing the status of real space and feedbacking valued information to real space after visual processing in virtual space, draws increasing attention in fields such as robotics, factory automation (FA), and entertainment.Object tracking, which includes detection and tracking, is one of the most important visual processing techniques for real-time applications.While different detection and tracking methods exist depending on the application, the keypoint matching-based detection method, which detects rigid objects, and the differential-based tracking method, which tracks the object with high accuracy, are important technologies in FA and robotics.On the other hand, the change in real space continues while visual processing is running in virtual space, so the visual processing with a large delay will introduce large errors, making the ultra-low delay visual processing critical to ensuring the validity of the result outputted from virtual space.Among various visual processing architectures, FPGA (Field-Programmable Gate Array)-based stream architecture is the one that is possible to achieve both complex spatial processing and delay lower than 1 ms while suitable algorithms and architectures are needed.Therefore, the FPGA-based ultra-low delay object tracking algorithm and architecture are required to ensure the validity of the tracking results when feedbacking the tracking results output from the object tracking process in the virtual space to the real space.FPGA-based stream architecture achieves ultra-low delay by performing visual processing while transmitting the image data from the sensor to the processor, which also means the processing for each image pixel is fully pipelined.The complex spatial processing in the object tracking algorithms has always been a problem, preventing object tracking from being implemented with fully pipelined architecture.Here shows three types of representative complex spatial processing.Spatial multi-search processing, which explores multiple possibilities corresponding to a factor and selects the best result among them to achieve robustness to the factor, exists in keypoint matching-based detection.When implementing it in FPGA with a fully pipelined style, many resources are required due to the extensive processing performed in the spatial domain for wide-range parameter search.Dependent sequential processing, which sequentially performs two dependent processing, exists in differential-based tracking.The dependent relation for two processing performed in a local spatial area generates not only high resource costs but also high delay.Spatial iterative processing, which performs the same processing iteratively with the output of the previous processing as the input, exists in differential-based tracking.Iterative processing of a large amount of spatial processing not only consumes expensive resources but also causes a significant delay in the FPGA implementation.On the other hand, the object moves along with being affected by disturbance while the tracking processing is running, resulting in the difference between the output state from virtual space
Author: Hu, Tingting
Language: Japanese
Open Access status: green
OpenAlex ID: W7025905001
FWCI:
Cites: 60
Cited by:
Related to: 10