Work

Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew’s Treatise

Year: 2024

Type: conference-paper

Abstract: This work presents a novel method for composing and improvising music inspired by Cornelius Cardew’s Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI’s ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI.

Source: IEEE International Conference on Big Data (BigData)

DOI: 10.1109/bigdata62323.2024.10825824

Authors: Tornike Karchkhadze, Keren Shao, Shlomo Dubnov

Institution: University of California San Diego

Language: English

Topic: Music Technology and Sound Studies

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: European Commission, University of California, San Diego, European Research Council

Award: G434337789