Year: 2025
Type: article
Abstract: The development of systems for the identification of elasmobranchs, including sharks and rays, is crucial for biodiversity conservation and fisheries management, as they represent one of the most threatened marine taxa. This challenge is constrained by data scarcity and the high morphological similarity among species, which limits the applicability of traditional supervised models trained on specific datasets. In this work, we propose an informed zero-shot learning approach that integrates external expert knowledge into the inference process, leveraging the multimodal CLIP framework. The methodology incorporates three main sources of knowledge: detailed text descriptions provided by specialists, schematic illustrations highlighting distinctive morphological traits, and the taxonomic hierarchy that organizes species at different levels. Based on these resources, we design a pipeline for prompt extraction and validation, taxonomy-aware classification strategies, and enriched embeddings through a prototype-guided attention mechanism. The results show significant improvements in CLIP’s discriminative capacity in a complex problem characterized by high inter-class similarity and the absence of annotated examples, demonstrating the value of integrating domain knowledge into methodology development and providing a framework adaptable to other problems with similar constraints.
DOI: 10.3390/make7040146
Institution: University of Alicante
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,172
Funder: European Commission
OpenAlex ID: W4416296006