Year: 2025
Type: preprint
Abstract: Grammar serves as a cornerstone in programming languages and software engineering, providing frameworks to define the syntactic space and program structure. Existing research demonstrates the effectiveness of grammar-based code representations in small-scale models, showing their ability to reduce syntax errors and enhance performance. However, as language models scale to the billion level or beyond, syntax-level errors become rare, making it unclear whether grammar information still provides performance benefits. To explore this, we develop a series of billion-scale GrammarCoder models, incorporating grammar rules in the code generation process. Experiments on HumanEval (+) and MBPP (+) demonstrate a notable improvement in code generation accuracy. Further analysis shows that grammar-based representations enhance LLMs' ability to discern subtle code differences, reducing semantic errors caused by minor variations. These findings suggest that grammar-based code representations remain valuable even in billion-scale models, not only by maintaining syntax correctness but also by improving semantic differentiation.
Source: arXiv (Cornell University)
Authors: Liang, Qingyuan, Zhang, Zhao, Sun, Zeyu, Lin, Zheng, Luo, Qi, Xiao, Yueyi, Chen, Yizhou, Zhang, Yuqun, Zhang, Haotian, Zhang, Lu, Chen, Bin, Xiong, Yingfei
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6979374139
Cites:
Cited by:
Related to: