Year: 2017
Type: dissertation
Abstract: In recent years it has become possible to train very deep neural networks. Deep convolutional neural networks (CNNs) have been able to extract relevant information from human faces. Deep fully convolutional neural networks (FCNNs) have been able to do dense pixel-per-pixel segmenting of images. This thesis explored whether it is possible to combine these two approaches and train an FCNN using non-realistic synthetic data to do dense pixel-per-pixel geometry tracking of real-world human faces. Training data was generated by rendering because suitable training dataset was not readily available. UV mapping of the underlying 3D model was used as a basis for the geometry mapping. Neural network topology was based on the U-net design, which is an FCNN with skip connections. Loss function was a simple L1 loss between result and target images. UV gradient images were used as an additional loss term. Data augmentation was used to expand the training dataset and to create occlusions. Various visualization methods were developed to help assess the accuracy of the generated geometry. Results were encouraging as the final network successfully generalized to real-world facial images. Faces were accurately segmented out of the backgrounds, and plausible geometry was generated inside them. Data augmentation prevented the network from overfitting. The occlusion augmentation method enabled the network to inpaint geometry under various obstructions, even ones the network had never seen. Temporal stability of the generated geometry was satisfactory but could be improved, especially under occlusions.
Source: Aaltodoc (Aalto University)
Author: Ronkainen, Mikko
Institution: Aalto University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7148207238
