Heterogeneous Low-Cost Distributed Probing for LLM Observation
Year: 2026
Type: preprint
Abstract: I describe a distributed probing protocol that runs a 100M-cell dense activation grid on thirteen open LLMs (124M to 7.6B parameters) using a small cluster of personal machines: a main workstation plus a few older laptops and an occasional accelerator, with a remote archive node holding the merged output. Calibration runs once on the host with enough RAM to hold the model; heat-fill runs in additive uint32 shards across whatever workers are available. The merge step uses element-wise sum and rejects shards with mismatched inputs by hashing. Wall-clock model: T = T_calib + T_fill * P / W for W workers on P probes. Heat tensor memory: |H| = L * d * Q * S * 4 bytes, where the product is at least 1e8. On commodity CPU at P=1200 and bfloat16, a 7B-class model completes in approximately one hour of calibration plus four worker-hours of heat-fill, all within ~400 MB of dense grid memory. No cloud burst. No data-center GPU.
Author: Chen, Ho Yiing
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7162780775
Cites:
Cited by:
Related to: