Work

Visual Conceptual Blending with Large-scale Language and Vision Models

Year: 2021

Type: preprint

Abstract: We ask the question: to what extent can recent large-scale language and image generation models blend visual concepts? Given an arbitrary object, we identify a relevant object and generate a single-sentence description of the blend of the two using a language model. We then generate a visual depiction of the blend using a text-based image generation model. Quantitative and qualitative evaluations demonstrate the superiority of language models over classical methods for conceptual blending, and of recent large-scale image generation models over prior models for the visual depiction.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2106.14127

Authors: Ge, Songwei, Parikh, Devi

Institutions: University of Maryland, College Park, Georgia Institute of Technology, Meta (United States)

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W3175732208

Cites: 32

Cited by: 13

Related to: 10