Enhancing Lifelog Question-Answering with Retrieval-Augmented Generation and Reasoning Large Language Models
Year: 2026
Type: dissertation
Abstract: Lifelogging refers to the process of passively collecting, storing, and analysing an individual’s daily life data from wearable cameras, sensors, and mobile devices. Question-answering (QA) on lifelog data involves posing questions on lifelog data and generating answers in natural language. Lifelog QA is a compelling yet challenging problem due to the extensive volume and the multimodal and temporal nature of lifelogging. Addressing this task requires a retriever to find relevant information within the lifelog collection before generating answers. With recent advancements in large language models (LLMs), their capabilities in QA tasks show potential to advance lifelog understanding. In addition, with their reasoning capability, LLMs can solve the aggregation question in lifelogging that requires contextual understanding and reasoning. This study investigates the lifelog QA task with RetrievalAugmented Generation and reasoning LLMs by proposing an end-to-end lifelog QA system. To achieve this, I first introduce a lifelog retrieval system to retrieve relevant information necessary for answering questions. A multi-modal reranking model is also developed to further improve the retrieval performance for lifelog QA. Subsequently, a RAG pipeline is developed to analyse the context from the retriever and generate the answers to the questions. Finally, reasoning LLMs for lifelog QA are also investigated to address complex aggregation questions. In summary, this thesis primarily contributes to the field with a QA system using RAG and reasoning LLMs to address the lifelog QA task.
Author: Tran, Quang-Linh
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7224691509