Year: 2023
Type: article
Abstract: Classical differential private DP-SGD implements individual clipping with random subsampling, \nwhich forces a mini-batch SGD approach. We provide a general differential private algorithmic \nframework that goes beyond DP-SGD and allows any possible first order optimizers (e.g., classical \nSGD and momentum based SGD approaches) in combination with batch clipping, which clips an \naggregate of computed gradients rather than summing clipped gradients (as is done in individual \nclipping). The framework also admits sampling techniques beyond random subsampling such as \nshuffling. Our DP analysis follows the f -DP approach and introduces a new proof technique based \non a slightly stronger adversarial model which allows us to derive simple closed form expressions \nand to also analyse group privacy. In particular, for E epochs work and groups of size g, we show a√gE DP dependency for batch clipping with shuffling.
Authors: Dijk, Marten, Nguyen, Phuong Ha, Nguyen, Toan, Nguyen, Lam
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7028857130