Work

Adversarial Auditing of Machine Learning Models under Compound Shift

Year: 2023

Type: conference-paper

Abstract: Machine learning (ML) models often perform differently under distribution shifts, in terms of utility, fairness, and other dimensions.We propose the Adversarial Auditor for measuring the utility and fairness performance of ML models under compound shifts of outcome and protected attributes.We use Multi-Objective Bayesian Optimization (MOBO) to account for multiple metrics and identify shifts where model performance is extreme, both good and bad.Using two case studies, we show that MOBO performed better than random and grid-based approaches in identifying scenarios by adversarially optimizing objectives, highlighting the value of such an auditor for developing fair, accurate and shift-robust models.

Source: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)

DOI: 10.14428/esann/2023.es2023-182

Authors: Karan Bhanot, Dennis Wei, Ioana Baldini, Kristin Bennett

Institutions: Rensselaer Polytechnic Institute, IBM (United States)

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

Open Access status: bronze

OpenAlex ID: W4386815481

FWCI:

Cites: 9

Cited by:

Related to: 10