Year: 2024
Type: conference-paper
Abstract: Question Answering (QA) establishes a natural and intuitive way for humans to interpret and understand multimodal sensor data. However, existing sensor-based QA systems are limited in the types of questions & answers, and the duration of sensor data they can handle. In this demo, we introduce an end-to-end QA system for long-term multimodal timeseries sensors powered by Large Language Models (LLMs). Our system features a novel pipeline with LLM-based question decomposition, sensor data query and LLM-based answer assembly. We further quantize the LLMs and deploy our system on two typical edge platforms, delivering higher-quality answers with low latency.
Authors: Xiaofan Yu, Lanxiang Hu, Benjamin Reichman, Rushil Chandrupatla, Dylan Chu, Xiyuan Zhang, Larry Heck, Tajana S Rosing
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Awards: FET: Small: Brain-Inspired Hyperdimensional Computing for IoT Applications, AI Institute for Learning-Enabled Optimization at Scale (TILOS), CC* NPEO: Toward the National Research Platform, CCRI: ABR: Cognitive Hardware and Software Ecosystem Community Infrastructure (CHASE-CI), Category II: A Prototype National Research Platform, Collaborative Research: MLWiNS: Hyperdimensional Computing for Scalable IoT Intelligence Beyond the Edge, Collaborative Research: IIS: RI: Medium: Lifelong learning with hyper dimensional computing, CCRI: ENS: Cognitive Hardware and Software Ecosystem Community Infrastructure (CHASE-CI)
OpenAlex ID: W4404034870