Work

Ishara: An AI-Powered Voice-Driven Browser for Accessible Web Interaction

Year: 2026
Type: preprint
Abstract: Ishara is an AI-powered, voice-controlled desktop browser that lets users navigate webpages, fill in forms, and get information through spoken commands instead of a mouse and keyboard. It is designed for people with motor, visual, hearing, or cognitive disabilities, including ADHD and dyslexia. Ishara pairs a conversational AI agent with access to the full DOM of the live page, so it can interpret user intent and carry out multi-step web tasks on its own. It is organized into five layers and built with Electron.js, Puppeteer, GPT-4o, Whisper, and Sarvam TTS. Accessibility modules include an ADHD focus mode, dyslexia-friendly rendering, multilingual voice interaction (English, Hindi, Marathi), AI page summarization, and an offline, in-browser 3D Indian Sign Language (ISL) avatar that signs YouTube captions and chatbot responses. In an informal pilot with five college peers, Ishara executed 44 of 50 voice commands successfully (88%), with 80-90% success across navigation, search, scrolling, playback control, and page interaction.
Language: English
Open Access status: green
OpenAlex ID: W7225998628
Cites:
Cited by:
Related to: 10