A Hybrid Multimodal Siamese Network for Speech-based Kinship Verification Leveraging the Kin-FVs Dataset
Year: 2026
Type: article
Abstract: The process of determining whether two individuals belong to same family or not called kinship verification, has been widely studied using facial images and video data, but is relatively underexplored in the audio domain.To address this problem, this paper introduces the Kinship Families Voices (Kin-FVs) dataset, a novel audio-only corpus specifically designed for kinship verification, comprising 564 recordings of 94 individuals across 25 real families spanning eight languages.Also propose a hybrid multimodal Siamese network that fuses log-mel spectrogram features extracted via ResNet50 with contextual self-supervised embeddings from Wav2Vec2.0,using intermediate pairwise fusion to learn kinship similarity.In addition to the proposed model, unimodal models (ResNet50 and SSL) and a late fusion are also evaluated, where the proposed framework achieves 93.0% accuracy and 6.2% EER on unseen test data across four parent-child relations, outperforming both unimodal and late-fusion methods.These findings highlight on the effectivity of multimodal learning for kinship verification based on speech signal, and this study provides a foundation with a new dataset and baselines.
Institution: University of Mosul
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: University of Mosul
OpenAlex ID: W7213644491