Work

AnyEnhance: A Unified Generative Model With Prompt-Guidance and Self-Critic for Voice Enhancement

Year: 2025

Type: article

Abstract: We introduceAnyEnhance, a unified generative model for voice enhancement that processes both speech and singing voices. Based on a masked generative model,AnyEnhanceis capable of handling both speech and singing voices, supporting a wide range of enhancement tasks including denoising, dereverberation, declipping, super-resolution, and target speaker extraction, all simultaneously and without fine-tuning.AnyEnhanceintroduces a prompt-guidance mechanism for in-context learning, which allows the model to natively accept a reference speaker’s timbre. In this way, it could boost enhancement performance when a reference audio is available and enable the target speaker extraction task without altering the underlying architecture. Moreover, we also introduce a self-critic mechanism into the generative process for masked generative models, yielding higher-quality outputs through iterative self-assessment and refinement. Extensive experiments on various enhancement tasks demonstrateAnyEnhanceoutperforms existing methods in terms of both objective metrics and subjective listening tests.

Source: IEEE Transactions on Audio Speech and Language Processing

DOI: 10.1109/taslpro.2025.3587393