How Small Can a Code Model Be? A Reproducible Study of One-Million-Parameter Python Synthesis with Compiler Feedback for Microcontrollers
Year: 2026
Type: preprint
Abstract: Can a language model small enough to live in the flash memory of a microcontroller write useful code? We study this question with a fully reproducible setup: a benchmark of IoT-oriented Python function-synthesis tasks with executable unit tests and audited train/test separation, decoder-only Transformers with 0.53 M and 1.05 M parameters, and a portable INT8 inference engine that runs unchanged on a PC, in a browser and on an ESP32-S3. On unseen wordings of tasks from the training families, the 1.05 M model passes the unit tests on 67.9 % of tasks with one-shot greedy decoding (mean of 3 seeds). Signature anchoring, which forces the answer to start with the function header copied from the request, raises this to 77.5 %. A compiler-in-the-loop repair procedure (Recursive Error-Feedback, REF) gated by compile() and a static undefined-name check reaches 79.8 %. REF mainly fixes syntax: it is no better than resampling under the same budget, and removing the error message from the repair prompt does not hurt. The 0.53 M model is at least as accurate (82.4 % with anchoring, s.d. 1.3) in 588,504 bytes of flash. On task families held out from training, accuracy falls to 1–2 %, and on MBPP to zero. Million-parameter models are therefore practical, verifiable specialists for a closed catalogue of embedded utility functions, not general programmers. Code, data, trained models and ESP32 firmware: https://github.com/rohithyarramala/microcoder (archived version 1.0.0: https://doi.org/10.5281/zenodo.24187363).
Author: Yarramala, Rohith
Institution: Defence Institute of Advanced Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7226025258
