Work

Using ChatGPT To Generate Walking Directions—Could It Potentially Help Visually Impaired With Micronavigation?

Year: 2025

Type: article

Abstract: Purpose: Micronavigation poses challenges for blind and visually impaired individuals (BVI). We investigated the potential of using ChatGPT4o to provide navigation directions. Methods: We created a dataset of real-world micronavigation scenarios consisting of 113 scene images and corresponding human-generated scene descriptions. A total of 407 human-generated wayfinding queries and their expected responses were used for evaluation (245 answerable and 162 not answerable based on the scenes). "I do not know" as a response was expected for the negative cases. Sensitivity (SEN) and specificity (SPE) of navigation directions output by ChatGPT4o was evaluated under different input conditions. Results: The default ChatGPT4o, with scene images as inputs, resulted in SEN and SPE values of 64% and 75.9%, respectively. Instructing via prompts on how to respond to unanswerable questions did not affect SEN, but increased SPE by 14 %. SEN and SPE both decreased by about 13% and 7%, respectively, when ChatGPT4o first generated text description from the input images and then answered the queries based the generated descriptions. When human-written scene descriptions were provided as input, SEN and SPE improved by about 17% and 16%, respectively. Conclusions: Some spatial information needed for micronavigation was diminished in generative scene descriptions. This problem may be addressed by one-step vision language models. General-purpose ChatGPT4o with fine-tuned prompts was still inferior to human in scene understanding. A multimodal large language model custom-trained on navigation scenarios might help improve the performance. Translational Relevance: AI chat-bots may have the potential to provide navigation assistance to BVI to a certain degree.

Source: Translational Vision Science & Technology

DOI: 10.1167/tvst.14.5.28

Authors: Junxian He, Shrinivas Pundlik, Gang Luo

Institutions: Massachusetts Eye and Ear Infirmary, Harvard University, Chongqing University

Language: English

Topic: Assistive Technology in Communication and Mobility

Subfield: Occupational Therapy

Field: Health Professions

Domain: Health Sciences

Open Access status: gold

Estimated APC paid: $1,850

OpenAlex ID: W4410817201