PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generation
Year: 2026
Type: conference-paper
Abstract: Podcast script generation requires LLMs to synthesize structured, context-grounded dialogue from diverse inputs, yet systematic evaluation resources for this task remain limited.To bridge this gap, we introduce PodBench, a benchmark comprising 800 samples with inputs up to 21K tokens and complex multi-speaker instructions.We propose a multifaceted evaluation framework that integrates quantitative constraints with LLM-based quality assessment.Extensive experiments reveal that while proprietary models generally excel, open-source models equipped with explicit reasoning demonstrate superior robustness in handling long contexts and multi-speaker coordination compared to standard baselines.However, our analysis uncovers a persistent divergence where high instruction following does not guarantee high content substance.PodBench offers a reproducible testbed to address these challenges in long-form, audio-centric script generation.The code and data are publicly accessible here.
Authors: Chenning Xu, Mao Zheng, Mingyu Zheng, Mingyang Song
Institution: Tencent (China)
Language: English
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166825614