Work

Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data

Year: 2023

Type: conference-paper

Abstract: Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains.However, these models are only accessible through a restricted API, creating barriers for new research and progress in the field.We propose a pipeline that can automatically generate a highquality multi-turn chat corpus by leveraging ChatGPT to engage in a conversation with itself.Subsequently, we employ parameter-efficient tuning to enhance LLaMA, an open-source large language model.The resulting model, named Baize, demonstrates good performance in multi-turn dialogues with guardrails that minimize potential risks.Additionally, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT.The Baize models and data are released for research purposes only. 1

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2023.emnlp-main.385

Authors: Canwen Xu, Daya Guo, Nan Duan, Julian McAuley

Institutions: University of California San Diego, Sun Yat-sen University, Microsoft Research Asia (China)

Language: English

Topic: Speech and dialogue systems

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W4389524372

FWCI: 23.31

Cites: 21

Cited by: 130

Related to: 10