Work

A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

Year: 2022

Type: preprint

Abstract: Vision-language models can encode societal biases and stereotypes, but there are challenges to measuring and mitigating these multimodal harms due to lacking measurement robustness and feature degradation. To address these challenges, we investigate bias measures and apply ranking metrics for image-text representations. We then investigate debiasing methods and show that prepending learned embeddings to text queries that are jointly trained with adversarial debiasing and a contrastive loss reduces various bias measures with minimal degradation to the image-text representation.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2203.11933

Authors: Berg, Hugo, Hall, Siobhan Mackenzie, Bhalgat, Yash, Yang, Wonsuk, Kirk, Hannah Rose, Shtedritski, Aleksandar, Bain, Max

Institution: University of Oxford

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Engineering and Physical Sciences Research Council, Economic and Social Research Council

Awards: G4489679334, G4605265282, EPSRC Centre for Doctoral Training in Autonomous Intelligent Machines and Systems, Grand Union Doctoral Training Partnership

OpenAlex ID: W4221140859

Cites:

Cited by: 10

Related to: 10