Year: 2026
Type: conference-paper
Abstract: Pretrained large language models (LLMs) are typically reused as indivisible artifacts, adapted, merged, or ensembled as a whole.In this study, we show that LLMs can instead be structurally recomposed as modular building blocks to create new architectures without access to original training data.We introduce architecturelevel reassembly as a new reuse paradigm, in which Transformer blocks from heterogeneous models are treated as reusable components.This idea is formalized through a twodimensional reassembly space that supports both vertical recombination across depth and horizontal composition within layers.To make this space tractable, we propose a chromosomebased architectural encoding and perform a bilevel multi-objective evolutionary optimization over vertical structure and horizontal composition.To resolve representation incompatibility across heterogeneous blocks, we introduce lightweight glue layers trained via data-free knowledge distillation, enabling valid information flow without modifying pretrained parameters.Our results demonstrate that architecturelevel reassembly unlocks a new dimension of flexibility in model reuse, pointing toward a modular and evolutionary view of LLM design.
Authors: Xingyu Wu, Yu Zhou, KC Tan
Institution: Hong Kong Polytechnic University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: Hong Kong Polytechnic University
Award: G3540495310
OpenAlex ID: W7166840267