Year: 2012
Type: dissertation
Abstract: Live video communications over bandwidth constrained ad-hoc radio networks necessitates high \ncompression rates. To this end, a model based video communication system that incorporates \nflexible and accurate 3D modelling and reconstruction is proposed in part. Model-based video coding \n(MBVC) is known to provide the highest compression rates, but usually compromises photorealism \nand object detail. High compression ratios are achieved at the encoder by extracting and transmit- \nting only the parameters which describe changes to object orientation and motion within the scene. \nThe decoder uses the received parameters to animate reconstructed objects within the synthesised \nscene. This is scene understanding rather than video compression. 3D reconstruction of objects \nand scenes present at the encoder is the focus of this research. \n3D Reconstruction is accomplished by utilizing the Patch-based Multi-view Stereo (PMVS) frame- \nwork of Yasutaka Furukawa and Jean Ponce. Surface geometry is initially represented as a sparse \nset of orientated rectangular patches obtained from matching feature correspondences in the input \nimages. To increase reconstruction density these patches are iteratively expanded, and filtered \nusing visibility constraints to remove outliers. Depending on the availability of segmentation in- \nformation, there are two methods for initialising a mesh model from the reconstructed patches. \nThe first method initialises the mesh from the object's visual hull. The second technique initialises \nthe mesh directly from the reconstructed patches. The resulting mesh is then refined by enforcing \npatch reconstruction consistency and regularization constraints for each vertex on the mesh. \nTo improve robustness to outliers, two enhancements to the above framework are proposed. The \nfirst uses photometric consistency during feature matching to increase the probability of selecting \nthe correct matching point first. The second approach estimates the orientation of the patch \nsuch that its photometric discrepancy score for each of its visible images is minimised prior to \noptimisation. The overall reconstruction algorithm is shown to be flexible and robust in that it can reconstruct \n3D models for objects and scenes. It is able to automatically detect and discard outliers and may be \ninitialised by simple visual hulls. The demonstrated ability to account for surface orientation of the \npatches during photometric consistency computations is a key performance criterion. Final results \nshow that the algorithm is capable of accurately reconstructing objects containing fine surface \ndetails, deep concavities and regions without salient textures.
Author: Naren. Ramchunder
Language: English
Topic: Advanced Vision and Imaging
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2133834095