SAEP-SVD: Sign Alignment and Energy Preservation for Stable Distillation of Extreme High-Rank Models
Year: 2026
Type: preprint
Abstract: SAEP-SVD (Sign Alignment and Energy Preservation) introduces a novel matrix distillation framework designed for high-rank compression of Diffusion Transformers (DiT, Z-Image, FLUX) without feature collapse, bypassing the classical constraints of the Eckart–Young–Mirsky theorem. Key Technical Highlights: Sign & Phase Alignment (SA): Prevents phase cancellation during singular vector truncation by enforcing strict directional projection consistency across latent weights. Energy Preservation Dynamics (EP): Dynamically redistributes singular values to protect critical high-frequency features, fine linework, and subtle artistic "gestures". Extreme High-Rank Scaling (Rank 512–2048+): Enables stable distillation of ultra-heavy LoRA adapters down to Rank 256/128 while maintaining full stylistic fidelity. Feature Preservation in DiT: Eliminates the "washed out" artifacting common to standard L2/Frobenius SVD compression in modern transformer attention layers. Empirical Validation: Proves zero degradation in fine detail rendering, text alignment, and stylistic consistency when distilling high-rank Z-Image adapters compared to uncompressed source weights. GitHub Repository: https://github.com/OrakulStudio/SAEP-SVD-Beyond-Eckart-Young
Author: Nykonov, Roman
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221474475
Cites:
Cited by:
Related to: