Year: 2025
Type: conference-paper
Abstract: We know from prior work that LLMs encode social biases, and that this manifests in clinical tasks (Gerszberg, 2024;Zack et al., 2024;Zhang et al., 2020).In this work we adopt tools from mechanistic interpretability to unveil sociodemographic representations and biases within LLMs in the context of healthcare.Specifically, we ask: Can we identify activations within LLMs that encode sociodemographic information (e.g., gender, race)?We find that, in three open weight LLMs, gender information is highly localized in MLP layers and can be reliably manipulated at inference time via patching.Such interventions can surgically alter generated clinical vignettes for specific conditions, and also influence downstream clinical predictions which correlate with gender, e.g., patient risk of depression.We find that representation of patient race is somewhat more distributed, but can also be intervened upon, to a degree.To our knowledge, this is the first application of mechanistic interpretability methods to LLMs for healthcare 1 . 1 Our code is available at https://github.com/ hibaahsan/interp-healthcare-bias/ "Generate a clinical vignette of a patient with Multiple Sclerosis … "male" patch LLM ( ) "Is this patient at risk of depression?
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416034639