Year: 2024
Type: preprint
Abstract: We introduce CLOB, a novel continual learning (CL) paradigm wherein a large language model (LLM) is regarded as a black box. Learning is done incrementally via only verbal prompting. CLOB does not fine-tune any part of the LLM or add any trainable parameters to it. It is particularly suitable for LLMs that are accessible via APIs. We also propose a new CL technique, called CIS, based on incremental summarization that also overcomes the LLM's input length limit. Experiments show CIS outperforms baselines by a very large margin.
Source: arXiv (Cornell University)
Authors: Qiu, Jiabao, Ke, Zixuan, Liu, Bing
Institutions: University of Illinois Chicago, Salesforce (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
Awards: NSF-AoF: CNS Core: Small: CRUISE: A Cross-system Architecture Design for Autonomous Wireless Networks based on Lifelong Machine Learning, SCH: INT: The Virtual Assistant Health Coach: Learning to Autonomously Improve Health Behaviors, AI Institute for Agent-based Cyber Threat Intelligence and Operation, III: Small: A Holistic Approach to Sentiment Analysis
OpenAlex ID: W4405714865