Year: 2026
Type: preprint
Abstract: AI assistants on smart glasses need to know what surrounds the user and what the user is looking at. Obtaining this context typically relies on a world-facing camera, raising privacy concerns for bystanders. We present GlintMarkers, a system for gaze-anchored spatial perception using a single inward-facing eye camera. Our key observation is that the cornea acts as a mirror that encodes both gaze direction and visual information about the environment in a small, low-contrast reflection. To extract spatial and semantic information from this reflection despite the camera's limited pixel budget, we design passive retroreflective markers that concentrate reflected near-infrared light onto the cornea, producing bright glint patterns. We develop a custom Perspective-n-Point (PnP) estimation framework adapted to corneal imaging and perform orientation and distance estimation of marked objects, as well as unique object identification.
Source: arXiv (Cornell University)
Institution: Carnegie Mellon University
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7154448686
Cites:
Cited by:
Related to: