Work

Pupil Tracking Without Goggles: A Solar-Assisted TinyML Brim-Based Pupil and Head Motion Tracking System with Digital Mimic

Year: 2026

Type: article

Abstract: This work presents a solar-assisted wearable pupil-tracking microsystem that integrates on-device inference, head pose sensing, energy harvesting, and mobile visualization within a single hat-based platform. The device is fabricated using 3D-printed thermoplastic polyurethane (TPU), providing a flexible and durable structure that supports embedded electronics while maintaining user comfort. The brim houses two ESP32-S3 microcontrollers, two grayscale eye-facing cameras, an inertial measurement unit (IMU), a lithium-polymer battery, a custom-shaped flexible amorphous silicon photovoltaic module, and a charging circuit. Each microcontroller runs an INT8-quantized convolutional neural network (CNN) that performs real-time 2D pupil coordinate regression and eye closure classification from 64 × 64 grayscale images. Pupil coordinates, eye state information, and head pose measurements are transmitted via Bluetooth Low Energy (BLE) to a custom iOS application, where a Unity-based 3D avatar mirrors the wearer’s relative eye movement and head motion while enabling lightweight data logging for offline analysis. By relocating sensing, computation, and power electronics from eyewear to the hat brim, the proposed system enables unobtrusive, autonomous pupil and eye state monitoring for naturalistic cognitive and behavioral research without requiring personal point-of-gaze calibration.

Source: Micromachines

DOI: 10.3390/mi17101183

Authors: Robert Graziano, Linda Valitutti, Juliette Bassett, Anwar Elhadad

Institution: Binghamton University

Language: English

Topic: Gaze Tracking and Assistive Technology

Subfield: Human-Computer Interaction

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: gold

Estimated APC paid: $2,534