Evaluating human-like similarity biases at every scale in Large Language Models: Evidence from remote and basic-level triads.
Year: 2024
Type: article
Abstract: In the remote triad task, participants judge the relatedness between randomly chosen words in a three-alternative choice triadic judgement task. While most word pairs in these triads are weakly related, humans agree on which to choose. This is theoretically interesting as it contradicts previous claims that suggest that the notion of similarity is unconstrained in principle (e.g., Goodman, 1972}. Here, we present new evidence from GPT-4, showing that context-aware LLMs provide excellent predictions of this task. Moreover, the strength of this effect was even larger than that found for basic-level comparisons, which involve highly similar items. Together, this implies that the similarity of human representations is highly structured at every scale, even in tasks with limited context. Follow-up analysis provides insights into how LLMs are successful in this task. Further implications of the ability to compare words at every scale are discussed.
Author: De Deyne, Simon
Institution: The University of Melbourne
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W6990925860