Work

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability

Year: 2025

Type: conference-paper

Abstract: While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation.To promote further research on flowchartbased code generation, this work presents Flow2Code, a novel benchmark for flowchartbased code generation evaluation.The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode.Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly.Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models' performance.

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2025.findings-acl.425