POSTER: TinyLLM: Evaluation and Optimization of Small Language Models for Agentic Tasks on Edge Devices
Year: 2026
Type: conference-paper
Abstract: As autonomous AI agents increasingly rely on Large Language Models (LLMs) for decision making; the necessity for local, privacy preserving execution on edge devices has become paramount. This paper investigates the effectiveness of Small Language Models (SLMs) with equal or fewer than 4 billion parameters for agentic tasks, specifically function and API calling. Using the Berkeley Function Calling Leaderboard (BFCL), we provide a comprehensive evaluation of models including xLAM-2, Qwen3, and TinyLlama. We introduce a Direct Preference Optimization (DPO) pipeline that leverages AgentBank (ALFRED) trajectories to align SLMs toward expert level tool use behavior. Our experimental results demonstrate a clear performance hierarchy where models in the 1-3B range significantly outperform sub 1B variants, achieving up to 65.74% overall accuracy and 55.62% in multi turn scenarios. Furthermore, we validate deployment feasibility through 4-bit quantization and hardware simulation, showing that these models can operate within the 8GB VRAM constraints of an Nvidia Jetson Nano while maintaining a functional throughput of ∼ 2.6 tokens/sec.
Institutions: Clark Atlanta University, United International University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166350919