Work

Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent

Year: 2022

Type: conference-paper

Abstract: of decreasing noise magnitudes can improve loss at the final iteration, and yet theoretical understandings of the effectiveness of such schedules and their connections to optimization algorithms remain limited. In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions. We first present a dynamic noise schedule minimizing the utility upper bound of PGD, and show how the noise influence from each optimization step collectively impacts utility of the final model. Our study also reveals how impacts from dynamic noise influence change when momentum is used. We empirically show the connection exists for general non-convex losses, and the influence is greatly impacted by the loss curvature.

Source: ACM Conference on Fairness, Accountability, and Transparency (FAccT)

DOI: 10.1145/3531146.3533070

Authors: Junyuan Hong, Zhangyang Wang, Jiayu Zhou

Institutions: Michigan State University, The University of Texas at Austin

Language: English

Topic: Privacy-Preserving Technologies in Data

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, University of Texas at Austin, National Institute on Aging, Office of Naval Research

Awards: G3577022761, Identification of Mild Cognitive Impairment using Machine Learning from Language and Behavior Markers, G8006577219, CAREER: Harness the Big Data via Large-Scale Lifelong Learning, G8876996369