From Attention to Inference - A Technical Study of Large Language Models
Year: 2026
Type: article
Abstract: This paper presents a technical overview of the Transformer architecture and its role in modern Natural Language Processing (NLP). It examines the core components of the paradigm including self-attention mechanisms, tokenization, positional encoding, model families (encoder-only, decoder-only, and encoder–decoder), pretraining objectives, fine-tuning, and inference processes. System-level considerations such as KV caching, Time to First Token, throughput, and VRAM usage are also discussed. The paper is intended as a structured technical reference for practitioners and students working in machine learning and NLP.
Author: SIOUMPALAS, THOMAS
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7155609463
FWCI:
Cites:
Cited by:
Related to: