Tackling Attribute Fine-grainedness in Cross-modal Fashion Search with Multi-level Features
Year: 2021
Type: conference-paper
Abstract: Leveraging information across modalities can facilitate customers throughout their journey, especially in the fashion domain where the visual modality plays an important role. Fashion products have a variety of visual groups of attributes such as shapes, colors, patterns, etc. Every category is fine-grained, i.e., attributes within a category may be visually very similar, e.g., v-neck vs. round-neck. The fine-grainedness of fashion attributes makes cross-modal fashion retrieval more challenging. In this paper, we address the problem of attribute fine-grainedness in fashion cross-modal retrieval by leveraging multi-level feature representations. In particular, we replace the commonly used spatial segmentation approach with a multi-level feature approach. We compare our approach with state-of-the-art models in general and fashion cross-modal retrieval and evaluate it on the Fashion200K and Fashion-Gen datasets. We record a 43.4% relative increase in text-to-image retrieval and a 57.8% relative increase in image-to-text retrieval on the Fashion200K dataset and a 48.6% relative increase in text-to-image retrieval and a 67.2%relative increase in image-to-text retrieval on the Fashion-Gen dataset while reducing the number of model parameters by 70%when compared with the baselines.
Authors: Goei, K., Hendriksen, M., de Rijke, M.; id_orcid 0000-0002-1086-0202
Institutions: University of Amsterdam, Ahold Delhaize (Netherlands)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7053164774