Year: 2023
Type: conference-paper
Abstract: Language models have recently broken into the public consciousness with the release of the wildly popular ChatGPT.Commentators have argued that language models could replace search engines, make college essays obsolete, or even write academic research papers.All of these tasks rely on accuracy of specialized information which can be difficult to assess for non-experts.Using 10 domain experts across science and culture, we provide an initial assessment of the coherence, conciseness, accuracy, and sourcing of two language models across 100 expert-written questions.While we find the results are consistently cohesive and concise, we find that they are mixed in their accuracy.These results raise questions of the role language models should play in generalpurpose and expert knowledge seeking.
Authors: Denis Peskoff, Brandon Stewart
Institution: Princeton University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4385572185
