Work

Follow-Up Questions Improve Generative AI Output and User Experience: Working Towards a Collaborative Model of Human-AI Interaction

Year: 2024
Type: dissertation
Abstract: This research investigates the impact of Large Language Models (LLMs) generating follow-up questions in response to user requests for short (1-page) text documents.This dissertation will argue that there are clear benefits to LLMs asking follow-up questions, and engaging users in thought-provoking and context-clarifying dialog before producing documents and other outputs.Two experiments support this research, including a pilot study and a larger full study with an improved design based on insights from the pilot study.In both experiments, users interacted with a novel web-based AI system designed to ask follow-up questions.Users requested documents they would like the AI to produce.The AI then generated follow-up questions to clarify the user's needs or offer additional insights before generating the requested documents.After answering the questions, users were shown a document generated using both the initial request and the questions and answers, and a document generated using only the initial request.Users indicated which document they preferred and gave feedback about their experience with the questionanswering process.The findings of these experiments show clear benefits to question-asking both in document preference and in the qualitative user experience, and further show that users found more value in questions which were thought-provoking, open-ended, or offered unique insights into the user's request as opposed to simple information-gathering questions.These results point to the need to incorporate follow-up questions and collaborative dialog into LLMs as a part of the human / AI interaction experience.
Author: Tix, Bernadette Jzexoia
Language: English
Open Access status: green
OpenAlex ID: W7110704144
Cites: 60
Cited by:
Related to: 10