AI on the Edge: An Automated Pipeline for PyTorch-to-Android Deployment and Benchmarking
Year: 2025
Type: preprint
Abstract: On-device latency ultimately determines whether a vision model is deployable on smartphones, yet collecting real-device latency at scale is costly and fragile. We present AI on the Edge, an automated measurement pipeline and open dataset for large-scale delegate-resolved Android benchmarking. AI on the Edge converts a PyTorch model zoo to TensorFlow Lite in FP32 and full-integer INT8 precision, then benchmarks each model on physical phones with three forced execution paths: CPU (XNNPACK), GPU (TFLite GPU), and NPU (NNAPI), so every latency is attributable to a known backend. Using one workstation and five commodity phones, AI on the Edge measured 586 architectures and produced 5,826 model × device × precision records, comprising about 17 500 backend benchmarks and 351 600 timed inferences (586 × 5 × 3 × 2 × 20), in roughly 56 unattended device-hours (up to 201 models/h/phone). Campaigns include both end-to-end Stage A+Stage B runs and Stage B-only replay from prebuilt TFLite artifacts, with identical on-device benchmark settings. The pipeline is designed for unattended operation via USB-reconnection handling, thermal cool-down scheduling, and a resumable processing ledger. We release the full per-run dataset with documented schema, public model/code resources, and a reproducibility harness that regenerates headline statistics from raw records. AI on the Edge reduces the practical cost of real-device benchmarking and provides a reusable substrate for hardware-aware architecture research.
Source: Preprints.org
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4416615293
