Year: 2021
Type: dissertation
Abstract: Recent advances in generic service robots have shown their introduction in various novel environments such as domestic and healthcare facilities. The CHARMIE (Collaborative Home/Healthcare Assistant Robot by Minho Industrial Electronics) project is an anthropomorphic robot that performs generic service tasks in non-standardized environment settings using machine learning algorithms, which allow the robot to make rational decisions based directly on the surrounding environment. In order to ensure an enhanced interaction between the robot and its specified users, this dissertation presents a real-time multi-stage deep learning pipeline for facial recognition algorithm. By detecting which of the pre-trained users the robot is interacting with, it can adapt its actions to best fit the user's needs. The multi-network system is composed of three modules. A multi-stage cascaded convolutional network searches for facial landmarks, such as eyes, nose, or mouth, predicting its bounding box coordinates, detecting all the faces showcased in the camera frame. An Inception-ResNet deep convolutional network was trained on a large dataset for facial feature extraction, comparing different hyperparameter settings and fine-tuning. That generates the feature vectors and provides an amplified network for facial recognition and an SVM classifier to categorize each of the faces recognized to the correct user. The combination of the three modules allowed an end-to-end facial detection and recognition that can be used as a real-time user-based identification method. The resulting method was furthermore implemented on the general service robot CHARMIE. Those tasks were carried out on students and teachers from the Laboratory of Automation and Robotics (LAR) of Minho University. For that, a dataset containing those people was built using images fetch from individual videos, with random frames being rotated or flipped to create a more robust system.
Author: Pereira, Nuno Miguel Azevedo
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7061593198