Year: 2026
Type: preprint
Abstract: A bstract Learned protein representations contribute to the broad utility of pretrained protein language models (PLMs), often enhanced by task-specific fine-tuning where parameter-efficient techniques are imperative due to large network size. Our work takes an orthogonal approach to such fine-tuning: rather than optimizing the network weights, we formulate adaptation as an optimization task over the parameterized activation space. Motivated by the prior work on curvature-tuning, we propose UPLiFT, U ncertainty P arameterized activation for Li ghtweight F ine- T uning, with enhanced adaptation capacity when the underlying model—whether pretrained or randomly initialized—remains fixed, for significantly improved parameter efficiency. Furthermore, its learnable non-linearity naturally allows to model uncertainty over the activation space through fine-tuning in a variational inference framework. Evaluation across multiple protein datasets shows that UP-LiFT performs competitively while requiring 3 − 11 fewer trainable paramters than weight-only adaptations. Bayesian UPLiFT further improves calibration while largely preserving the predictive performance of deterministically fine-tuned models. The narrow predictive performance margin with the low-rank adaptation baseline in residue-level tasks highlights the potential of our proposed approach as a parameter-efficient fine-tuning technique for PLM-based predictors.
Institutions: Texas A&M University, Brookhaven National Laboratory
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7221565446