Streaming Piano Transcription Based on Consistent Onset and Offset Decoding With Sustain Pedal Detection
Year: 2024
Type: conference-paper
Abstract: This paper describes a streaming audio-to-MIDI transcription method that can sequentially translate a piano recording into a sequence of note-on and note-off events. The sequence-to-sequence learning nature of this task may call for using a Transformer model, which has been used for offline transcription and could be extended for streaming transcription with a causal restriction of the attention mechanism. We assume that the decoder of this model suffers from the performance limitation. Although time-frequency features useful for onset detection are considerably different from those for offset detection, the single decoder is trained to output a mixed sequence of onset and offset events without guarantee of the correspondence between the onset and offset events of the same note. To overcome this limitation, we propose a streaming encoder-decoder model that uses a convolutional encoder aggregating local acoustic features, followed by an autoregressive transformer decoder detecting a variable number of onset events and another decoder detecting the offset events of the active pitches with validation of the sustain pedal at each time frame. Experiments using the MAESTRO dataset showed that the proposed streaming method performed comparably with or even better than the state-of-the-art offline methods while significantly reducing the computational cost.
Authors: Weixing Wei, Jiahao Zhao, Yulun Wu, Kazuyoshi Yoshii
Language: English
Topic: Music and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6949261906
FWCI:
Cites:
Cited by:
Related to: