Enhancing_language_model_adaptation_through_hierarchical_graph_based_knowledge_integration_for_technical_domains (proof of concept)
Year: 2026
Type: book
Abstract: This notebook introduces and evaluates a novel hierarchical architecture that integrates multiple knowledge sources—classical fine-tuning, ontological graphs, dynamic semantic graphs, and Self-Organizing Maps (SOM)—to enhance LLM adaptation for technical domains. We propose a 4-level gated architecture and conduct a systematic scaling study across datasets ranging from 500 to 3,500 sentences, using perplexity, generation diversity, and technical coherence as metrics. Our ablation analysis identifies the ontological graph as the single most valuable component, contributing a 6.5% perplexity reduction. However, we also identify a critical trade-off: models with strong knowledge integration exhibit elevated repetition rates in generated text (up to 37.4%), pointing to a key area for future research. Notably, our analysis reveals a trend suggesting that the full model may outperform the baseline with larger datasets, though this remains a preliminary observation requiring validation on substantially larger corpora (>10,000 samples). Our work demonstrates that while complex hierarchical adaptation architectures show promise, their benefits are not uniform and depend critically on dataset size. We release our code and data to facilitate reproducibility and further research.
Author: Shaykhulova, Aygul
Institution: Ufa University of Science and Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166340359