Work

Iterative Tilting for Diffusion Fine-Tuning

Year: 2025
Type: preprint
Abstract: We introduce iterative tilting, a gradient-free method for fine-tuning diffusion models toward reward-tilted distributions. The method decomposes a large reward tilt $\exp(λr)$ into $N$ sequential smaller tilts, each admitting a tractable score update via first-order Taylor expansion. This requires only forward evaluations of the reward function and avoids backpropagating through sampling chains. We validate on a two-dimensional Gaussian mixture with linear reward, where the exact tilted distribution is available in closed form.
Language: English
Open Access status: green
OpenAlex ID: W4417026712
Cites: 19
Cited by: 5
Related to: 10