Year: 2024
Type: conference-paper
Abstract: This study addresses the critical need for quantizing audio diffusion models to enable efficient synthesis on low-resource-constrained devices. We propose to design objective audio quality metrics for realism and fidelity, which can be utilized to optimize the denoising process of diffusion models. The current state of our research includes the application of quantization strategies from the image domain to off-the-shelf audio diffusion models such as AudioLDM and Make-an-Audio. The proposed work consists of exploring the specific operations in the U-Net architecture that should be quantized, focusing on weights and activations and how to effectively calibrate them. We plan to evaluate our framework on mobile devices to cater to a range of hardware- off the shelf and customized.
Author: Jayneel Vora
Institution: University of California, Davis
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4404373453