Work

FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings

Year: 2023

Type: conference-paper

Abstract: Federated learning is a training paradigm that learns from multiple distributed users without aggregating data on a centralized server, promising the ability to deploy machinelearning to a diverse population of users without first collecting large, labeled datasets.As federated learning involves averaging gradient updates across a decentralized population, there is a growing need for personalization of federated learning systems (i.e.conversational agents must personalize to individual users and the context of an interaction).In this work, we propose a new direction for personalization research within federated learning, leveraging both personal embeddings and shared context embeddings.We also present an approach to predict these "preference" embeddings, enabling personalization without backpropagation.Compared to state-of-the-art personalization baselines, our approach achieves a 50% improvement in test-time perplexity using 0.001% of the memory required by baseline approaches, and achieving greater sample-and compute-efficiency.

Source: Findings of the Association for Computational Linguistics: EACL

DOI: 10.18653/v1/2023.findings-eacl.64

Authors: Andrew Silva, Pradyumna Tambwekar, Matthew Gombolay

Institution: Georgia Institute of Technology

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Office of Naval Research

Awards: CPS: Medium: Learning-Enabled Assistive Driving: Formal Assurances during Operation and Training, G5350327428, AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups (AI-CARING), G8876996369