Year: 2026
Type: conference-paper
Abstract: We present ViLegalLM, comprising ViLegal-BERT and ViLegalQwen, the first suite of Vietnamese pretrained language models for legal text understanding and generation.It includes one encoder-only model (ViLegal-BERT, 135M parameters) and two decoderonly models (ViLegalQwen2.5-1.5B-Base and ViLegalQwen3-1.7B-Base),all continually pretrained on a newly curated 16GB Vietnamese legal corpus, significantly larger than previous work.To mitigate data scarcity, we construct three synthetic datasets using LLM-based generation and hard negative mining for True/False QA, Multiple Choice QA, and Natural Language Inference.We establish state-of-theart results among open-source models on four main Vietnamese legal downstream tasks spanning ten benchmarks, demonstrating that continual pretraining from base models consistently outperforms instruction-tuned adaptation.
Institution: Hung Yen University of Technology and Education
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166885101