Year: 2007
Type: dissertation
Abstract: ENGELSK:\nClassification of the audio track, of a motion picture, into traditional audio classes is a\nchallenging task. The reason for this is the amount of mixed content. Speech has often\nbackground music or environmental sounds, and music has often background environmental\nsounds or speech. Traditional methods for separating clear audio classes\nhave limited performance on mixed audio content. New methods and tools for automatic\nclassification of this type of audio are therefore needed. This project investigates\ncombinations of low level descriptors, dimensionality reduction by PCA and classification\nby KNN. A feature set consisting of Audio Power (AP), Audio Wave Form (AWF), Root-\nMean-Square (RMS), Short Time Energy (STE), Low Short-Time Energy Ratio, Zero-\nCrossing Rate (ZCR), High Zero-Crossing Rate Ratio (HZCRR) in the time domain and\nAudio Spectrum Centroid (ASC), Fundamental Frequency (FuF), Mel, Frequency Cepstral\nCoefficients (MFCC), Spectrum Flux (SF) in the frequency domain is extracted on\n30ms windows and integrated over a 1.2 second frame to yield a 23-dimmensional feature\nvector. The most suitable combination for separating speech from background music\nwere; AP, ASC, AWF, STE, RMS, SF, fourth MFCC, AP(min scalar), ZCR(min scalar) and\nSTE(min scalar). The combination has a majority of descriptors from the time domain.\nMost suitable combination of LLDs to separate speech with background environmental\nsounds from clear environmental sounds was found to be the 1th, 4th, 5th, 6th, 8th and\n9th Mel Frequency cepstran coefficients. MFCC is in the frequency domain. Best results\nwere achieved when the PCA returned 3 dimensions, and when the KNN classified the\nsamples based on the 4 closest neighbors. The results from testing different mixtures of\nspeech and music, to find the boundary where speech with background music no longer\nis categorized as speech, showed that the music signal had to be minimum 8 dB below the\nspeech signal to be classified as speech. Some of the low level descriptors which traditionally\nperforms well when separating clear classes, performed poorly in the experiments\nwith mixed classes. Especially ZCR and FuF failed to separate speech with background\nmusic from clear music. A final experiment classifies the audio track from the motion\npicture ’Groundhog Day’. First 80 percent of the movie is used for training of the KNN,\nand the remaining 20 percent was classified. After post processing the result was 76.9\npercent correctly classified. A table of content was then based on this classification.
Author: Eirik Gustavsen
Language: English
Topic: Music and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W93454604