Work

Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution

Year: 2025

Type: preprint

Abstract: Measuring task relatedness and mitigating negative transfer remain a critical open challenge in Multitask Learning (MTL). This work extends data attribution -- which quantifies the influence of individual training data points on model predictions -- to MTL setting for measuring task relatedness. We propose the MultiTask Influence Function (MTIF), a method that adapts influence functions to MTL models with hard or soft parameter sharing. Compared to conventional task relatedness measurements, MTIF provides a fine-grained, instance-level relatedness measure beyond the entire-task level. This fine-grained relatedness measure enables a data selection strategy to effectively mitigate negative transfer in MTL. Through extensive experiments, we demonstrate that the proposed MTIF efficiently and accurately approximates the performance of models trained on data subsets. Moreover, the data selection strategy enabled by MTIF consistently improves model performance in MTL. Our work establishes a novel connection between data attribution and MTL, offering an efficient and fine-grained solution for measuring task relatedness and enhancing MTL models.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2505.21438

Authors: Tu, Yiwen, Liu, Ziqi, Ma, Jiaqi W., Tang, Weijing

Institutions: Carnegie Mellon University, University of Illinois Urbana-Champaign

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Award: Statistical Learning and Inference for Network Data with Positive and Negative Edges

OpenAlex ID: W4415037518

Cites:

Cited by:

Related to: