Work

Analyzing AI-Simulated Multi-Agent Discussions: Extracting Structured Insights from LLM-Generated Dialogues

Year: 2025
Type: dissertation
Abstract: The proliferation of large language models (LLMs) has enabled the development of sophisticated multi-agent systems that can simulate realistic conversations.However, analyzing these complex, largely unstructured dialogues to extract meaningful insights is a significant challenge.This thesis presents a multifaceted framework for analyzing AI-generated multi-agent discussions, integrating natural language processing techniques, machine learning models, graph network analysis, and dialoguespecific metrics.Following the Design Science Research (DSR) methodology, the framework combines Sentence-BERT embeddings and RoBERTa sentiment classification to extract multidimensional insights from agent interactions.The framework's performance was evaluated using the DailyDialog dataset through a systematic baseline comparison across multiple analytical dimensions.The results demonstrate that the framework effectively identifies sentiment evolution, coherence patterns, thematic transitions, agent influence relationships, and relative agent credibility in the conversation.While the framework's accuracy may vary in dialogues with atypical structures, low topic diversity, or ambiguous conversation contexts, it generates interpretable visualizations that make conversational dynamics accessible to anyone seeking to understand multi-agent interactions.The theoretical contribution of this research advances the understanding of conversation pattern dynamics in multi-agent systems.The framework's analytical depth and interpretability qualities suggest its usefulness as a practical tool for researchers and practitioners to synthesize real-world human-generated conversational data.The study also presents future research and development suggestions to enhance analytical ability in specific domains, such as policymaking, business, and education, thereby improving robustness and applicability in real-world environments.
Institution: Tampere University
Language: English
Open Access status: green
OpenAlex ID: W7155666772
Cites: 60
Cited by:
Related to: 10