Work

PSST! Prosodic Speech Segmentation with Transformers

Year: 2023

Type: conference-paper

Abstract: We develop and probe a model for detecting the boundaries of prosodic chunks in untranscribed conversational English speech.The model is obtained by fine-tuning a Transformer-based speech-to-text (STT) model to integrate the identification of Intonation Unit (IU) boundaries with the STT task.The model shows robust performance, both on held-out data and on out-of-distribution data representing different dialects and transcription protocols.By evaluating the model on degraded speech data, and comparing it with alternatives, we establish that it relies heavily on lexico-syntactic information inferred from audio, and not solely on acoustic information typically understood to cue prosodic structure.We release our model 1 as both a transcription tool and a baseline for further improvements in prosodic segmentation.

Source: Conference on Computational Natural Language Learning

DOI: 10.18653/v1/2023.conll-1.31

Authors: Nathan Roll, Calbert Graham, Simon Todd

Institutions: University of California, Santa Barbara, University of Cambridge

Language: English

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Université de Reims Champagne-Ardenne, University of California, Santa Barbara

OpenAlex ID: W4389521026