Year: 2025
Type: article
Abstract: This research investigates the ability of large language models (LLMs) to recognize due process issues. Due process is a legal concept focused on the protection of the individual during interactions with government when life, liberty, or property are being impacted. Due process presents both substantive and procedural aspects that are challenging to incorporate into generative artificial intelligence. Through assessing model performance, creating benchmarking techniques, retrieval-augmented generation (RAG), and fine-tuning, this work seeks to measure due process recognition performance and improve performance in identifying due process issues. The results of evaluating larger parameter LLMs such as from Google, Meta, and OpenAI demonstrate that there are open-source and proprietary models capable of reliably recognizing due process issues. Low parameter models tested in this work displayed an inability to reliably recognize due process issues. Tuning and RAG were shown to improve due process recognition in low parameter models by nearly ten times, however, this work indicates that low parameter models still require significant improvement before being useful in due process recognition applications. This work also displays the utility of combining vector embedding techniques, LLM summarization, and clustering for finding patterns throughout time in large quantities of judicial opinions. Lastly, the benchmark created in this research displays a machine-based approach for quickly measuring due process recognition performance without a manual human evaluation that is within 10% of attorney scoring.
Author: Johnson, Joshua Paul
Institution: University of Louisville
Language: English
Subfield: Law
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7135872559