Objestures: Everyday Objects Meet Mid-Air Gestures for Expressive Interaction
Year: 2026
Type: conference-paper
Abstract: Everyday object-based interactions (EOIs) and mid-air gesture interactions (MAIs) have been widely explored, yet prior work on their integration often targets narrow use cases or specific technologies, leaving designers and developers with limited guidance that generalizes across diverse EOIs and MAIs. We introduce Objestures (“Obj” + “Gestures”)—five interaction types spanning EOIs and MAIs, forming a design space for expressive uni- and bimanual interaction. To evaluate the usefulness of Objestures, we conducted an exploratory user study (N = 12) on basic 3D tasks (rotation and scaling), which showed performance comparable to the headset’s native freehand manipulation. To understand the user experience, we conducted case studies with the same participants across three applications (Sound, Draw, and Shadow), where participants found the interactions intuitive, engaging, and expressive, and indicated interest in everyday use. We further demonstrate the potential of Objestures across diverse contexts through 30 examples, and discuss limitations and implications.
Authors: Zhuoyue Lyu, Per Ola Kristensson
Institution: University of Cambridge
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4415150884