Deep image representations for instance search
Year: 2017
Type: dissertation
Abstract: We address the problem of visual instance search, which consists to retrieve all\nthe images within an dataset that contain a particular visual example provided to\nthe system. The traditional approach of processing the image content for this task\nrelied on extracting local low-level information within images that was “manually\nengineered” to be invariant to di↵erent image conditions. One of the most popular\napproaches uses the Bag of Visual Words (BoW) model on the local features to\naggregate the local information into a single representation. Usually, a final reranking stage is included in the pipeline to refine the search results. Since the\nemergence of deep learning as the dominant technique in computer vision in 2012,\nmuch research attention has been focused on deriving image representations from\nConvolutional Neural Networks (CNN) models for the task of instance search as a\n“data driven” approach to designing image representations. However, one of the main\nchallenges in the instance search task is the lack of annotated datasets to fit CNN\nmodels parameters.\nThis work explores the capabilities of descriptors derived from pre-trained CNN\nmodels for image classification to address the task of instance retrieval. First, we\nconduct an investigation of the traditional bag of visual words encoding on local\nCNN features to produce a scalable image retrieval framework that generalizes well\nacross di↵erent retrieval domains. Second, we propose to improve the capacity of the\nobtained representations by exploring an unsupervised fine-tuning strategy that allow\nus to obtain better performing representations at the price of losing the generalization\nof the representations. Finally, we propose using visual attention models to weight\nthe contribution of the relevant parts of an image to obtain a very powerful image\nrepresentation for instance retrieval without requiring the construction of a large\nand suitable training dataset for fine-tuning CNN architectures.
Author: Eva Mohedano Robles
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2806690332