Extremely economical: how key frames affect consonant perception under different audio-visual skews
Year: 2006
Type: article
Abstract: line spaceIn audio-visual telecommunication, low video frame rates represent a popular method for saving on bandwidth requirements.When key frames displayed the extremes of lip movements we found that participants performed comparably to standard displays at 30 frames per second.Experiments were conducted to compare the effectiveness of a small number of algorithmically chosen key frames -typically 7 to 8 frames per second (fps) -to 30fps displays where audio and video were out of synch by as much as 233ms.Noised non-sense words like 'abagava' were presented to 20 participants who were asked to identify the middle consonant.The results indicate that key frame displays are as effective as 30fps when audio lags video by 87 to 167ms.Despite the low temporal resolution and varying exposure lengths, participants were able to integrate the given bi-modal information as well as the 30fps condition if the audio channel lagged the video by 87ms.The latter is recognized as being within the region of optimal audio-visual (AV) integration.
Authors: Hendrik Knoche, Hermann de Meer, David Kirsh
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W207283431