Work

Active learning for image retrieval via visual similarity metrics and semantic features

Year: 2024

Type: article

Abstract: We introduce an active learning framework for content-based image retrieval for video surveillance that can be trained ad-hoc for a single camera in a matter of minutes. This technique allows searching for both, known and unknown objects, given a region of interest. The process does not require prior labelled data and treats image retrieval as a binary classification task, in which frames can be similar or different from a query image. The technique is compatible with any pre-trained deep feature extractor. In addition, we propose a novel label propagation algorithm that benefits from (1) visual similarity of image pairs and (2) the semantic representation of the feature vectors from a pre-trained deep feature extractor. This approach allows to reduce the amount of labels needed, while avoiding the propagation of errors. Our experiments with three use-cases from a nuclear facility show the validity of the proposed method, which achieves high precision and recall while requiring minimal amounts of labelled data. • Present a relevant video frame retrieval pipeline that can be trained interactively. • Propose an active learning algorithm that leverages visual and semantic similarity. • Visual similarity makes label propagation less prone to introducing errors.

Source: Engineering Applications of Artificial Intelligence

DOI: 10.1016/j.engappai.2024.109239

Authors: Alvaro Casado-Coscolla, Carlos Sanchez-Belenguer, Erik Wolfart, Carlos Angorrilla-Bustamante, Vitor Sequeira

Institution: Joint Research Centre

Language: English

Topic: Image Retrieval and Classification Techniques

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,270

Funder: Joint Research Centre