Towards philosophical reasoning with agentic LLMs: Socratic method for scientific assistance
Year: 2025
Type: article
Abstract: As large language models (LLMs) become central tools in science, improving their reasoning capabilities is critical for meaningful and trustworthy applications. We introduce a Socratic agent for scientific reasoning, implemented through a structured system prompt that guides LLMs via classical principles of inquiry. Unlike typical prompt engineering or retrieval-based methods, our approach leverages definition, analogy, hypothesis elimination, and other Socratic techniques to generate more coherent, critical, and domain-aware responses. We evaluate the agent across diverse scientific domains and benchmark it on the abstraction and reasoning corpus challenge dataset, achieving 97.15% under a fixed prompting protocol and without fine-tuning or external tools. Expert evaluation shows improved reasoning depth, clarity, and adaptability over conventional LLM outputs, suggesting that structured prompting rooted in philosophical reasoning can improve the scientific utility of language models.
Authors: Hassan Harb, Yunkai Sun, Mustafa Unal, Abhishek Aggarwal, Chiara Bissolotti, Isik Su Buyuker, Sungil Hong, Luke R Johnson, Lateef Jolaoso, Bratin Sengupta, Michael Stuhr, Zhenzhen Yang, Brian J Ingram, Rajeev Surendran Assary
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,000
OpenAlex ID: W7108340679