FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks
Year: 2026
Type: article
Abstract: Low-rank adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet uniform rank assignment ignores the varying contribution of layers to a downstream task. Adaptive methods such as AdaLoRA allocate ranks via gradient and singular-value importance, but they do not exploit the representation change induced by the task. We introduce FLowRA, a calibration-based rank-allocation framework that combines a lightweight flow-change descriptor (velocity, acceleration, and directional alignment of hidden representations) with gradient activity to solve a constrained integer-budget problem. FLowRA complements existing LoRA variants, requires only two user-configurable parameters, and supports multiple architectures through architecture-specific implementations. On GLUE, SuperGLUE, instruction-following, code, vision, and vision-language benchmarks, FLowRA consistently outperforms uniform LoRA, DoRA, and AdaLoRA under low trainable-parameter budgets, reducing the gap to full fine-tuning by 48.5% on GLUE relative to the strongest baseline. We compare FLowRA with AdaLoRA using sensitivity-score correlations and report component ablations to assess the contribution of flow-guided allocation.
DOI: 10.3390/make8110389
Authors: Aleksei Samarin(Conceptualization, Investigation, Methodology, Project administration, Resources, Supervision, Writing – review & editing), Artem Nazarenko(Investigation, Methodology, Resources, Software, Writing – original draft), Egor Kotenko(Data curation, Formal analysis, Project administration, Visualization, Writing – original draft, Writing – review & editing), Aleksei Toropov(Methodology, Software, Visualization), Dmitry Nazarenko(Investigation, Software, Writing – original draft), Julia Afonina(Formal analysis, Investigation, Writing – original draft), Aleksandra Dozortseva(Investigation, Validation), Alexander Savelev(Investigation, Software, Validation), Elena Mikhailova(Data curation, Formal analysis, Validation), Alexander Motyko(Data curation, Resources), Valentin Malykh(Data curation, Formal analysis, Validation)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7236675494
