Year: 2010
Type: dissertation
Abstract: The inherently linear and sequential property of speech raises the need for\nways to better navigate through spoken documents. The strategy of navigation I\nfocus on in this thesis is summarization, which aims to identify important excerpts\nin spoken documents.\n\nA basic characteristic that distinguishes speech summarization from traditional\ntext summarization is the availability and utilization of speech-related features.\nMost previous research, however, has addressed this source from the perspective of\ndescriptive linguistics, in considering only such prosodic features that appear in that\nliterature. The experiments in this dissertation suggest that incorporating prosody\ndoes help but its usefulness is very limited—much less than has been suggested in\nsome previous research. We reassess the role of prosodic features vs. features arising\nfrom speech recognition transcripts, as well as baseline selection in error-prone\nand disfluency-filled spontaneous speech. These problems interact with each other,\nand isolated observations have hampered a comprehensive understanding to date.\n\nThe effectiveness of these prosodic features is largely confined because of their\ndifficulty in predicting content relevance and redundancy. Nevertheless, untranscribed\naudio does contain more information than just prosody. This dissertation\nshows that collecting statistics from far more complex acoustic patterns does allow\nfor estimating state-of-the-art summarization models directly. To this end, we propose\nan acoustics-based summarization model that is estimated directly on acoustic\npatterns. We empirically determine the extent to which this acoustics-based model\ncan effectively replace ASR-based models.\n\nThe extent to which written sources can benefit speech summarization has\nalso been limited, namely to noisy speech recognition transcripts. Predicting the \nsalience of utterances can indeed benefit from more sources than raw audio only.\nSince speaking and writing are two basic ways of communication and are by nature\nclosely related to each other, in many situations, speech is accompanied with relevant\nwritten text. Richer semantics conveyed in the relevant written text provides\nadditional information over speech by itself. This thesis utilizes such information\nin content selection to help identify salient utterances in the corresponding speech\ndocuments. We also employ such richer content to find the structure of spoken\ndocuments—i.e., subtopic boundaries—which may in turn help summarization.
Source: TSpace (University of Toronto)
Author: Xiaodan Zhu
Institution: University of Toronto
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: University of Pennsylvania
OpenAlex ID: W2520734307