Work

On Disentanglement of Asymmetrical Knowledge Transfer for Modality-Task Agnostic Federated Learning

Year: 2024

Type: conference-paper

Abstract: There has been growing concern regarding data privacy during the development and deployment of Multimodal Foundation Models for Artificial General Intelligence (AGI), while Federated Learning (FL) allows multiple clients to collaboratively train models in a privacy-preserving manner. This paper formulates and studies Modality-task Agnostic Federated Learning (AFL) to pave the way toward privacy-preserving AGI. A unique property of AFL is the asymmetrical knowledge relationships among clients due to modality gaps, task gaps, and domain shifts between clients. This raises a challenge in learning an optimal inter-client information-sharing scheme that maximizes positive transfer and minimizes negative transfer for AFL. However, prior FL methods, mostly focusing on symmetrical knowledge transfer, tend to exhibit insufficient positive transfer and fail to fully avoid negative transfer during inter-client collaboration. To address this issue, we propose DisentAFL, which leverages a two-stage Knowledge Disentanglement and Gating mechanism to explicitly decompose the original asymmetrical inter-client information-sharing scheme into several independent symmetrical inter-client information-sharing schemes, each of which corresponds to certain semantic knowledge type learned from the local tasks. Experimental results demonstrate the superiority of our method on AFL than baselines.

Source: Proceedings of the AAAI Conference on Artificial Intelligence

DOI: 10.1609/aaai.v38i10.29010

Authors: Jiayi Chen, Aidong Zhang

Institution: University of Virginia

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: National Science Foundation

Awards: III: Medium: Collaborative Research: Mining and Leveraging Knowledge Hypercubes for Complex Applications, Collaborative Research: CCRI: New: A Scalable Hardware and Software Environment Enabling Secure Multi-party Learning, Collaborative Research: PPoSS: LARGE: Co-designing Hardware, Software, and Algorithms to Enable Extreme-Scale Machine Learning Systems, III: Small: Multimodal Machine Learning for Data with Incomplete Modalities, III: Medium: Knowledge-Guided Meta Learning for Multi-Omics Survival Analysis