Software Lip-Sync Metrics Do Not Predict Delivered Audio-Visual Synchronisation: A Photodiode and Line-Output Measurement of Talking-Head Video Playback Against ITU-R BT.1359-1
Year: 2026
Type: preprint
Abstract: AI-generated talking-head video is certified for lip synchronisation with file-based metrics. However, the governing ITU-R BT.1359-1 thresholds derive from viewer perception. This study examined whether the delivered offset can be measured physically with a stated uncertainty, what offset the systems deliver, and whether SyncNet lip-sync error scores, distance (LSE-D) and confidence (LSE-C) can predict it. AOLSS, the Acoustic–Optical Latency Sensing System, samples a photodiode on the rendered mouth region and a line-output tap simultaneously at 20 kHz, and removes calibrated chain latency per cell. Inter-channel uncertainty (0.0144 ms) is negligible beside 60 Hz frame quantisation (4.81 ms). Chain latency spanned −33.3 to +31.3 ms, changing sign between players. Over four systems, three players and two resolutions (24 cells, 3637 valid trials), median delivered offsets were −210.9 to −79.2 ms, with sound lagging in every cell. 60.9%–88.4% of trials exceeded the ITU-R detectability thresholds and 31.6%–59.4% the acceptability thresholds. LSE-D and LSE-C were uncorrelated with delivered-offset magnitude in every player–resolution cell (r = −0.066 to +0.083). SyncNet’s offset estimate agreed with the measurement only within limits of half-width 282.2 ms, 1.5 times the wider acceptability limit. File-based metrics do not predict delivered synchronisation, which must be measured at the output.
Source: Preprints.org
Authors: Chien-Ta Bruce Ho, Xiya LI
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225198704