Year: 2026
Type: conference-paper
Abstract: AI-generated content has evolved from monolithic models to modular workflows, particularly on platforms like ComfyUI, enabling customization in creative pipelines.However, crafting effective workflows requires great expertise to orchestrate numerous specialized components, presenting a steep learning curve for users.To address this challenge, we introduce ComfyUI-R1, the first large reasoning model for automated workflow generation.Starting with our curated knowledge bases, we construct long chain-of-thought (CoT) reasoning data, including node selection, workflow planning, and code-level workflow representation.ComfyUI-R1 is trained through a twostage framework: (1) CoT fine-tuning for cold start, adapting models to the ComfyUI domain;(2) reinforcement learning for incentivizing reasoning capability, guided by a fine-grained rulemetric hybrid reward, ensuring format validity, structural integrity, and node-level fidelity.Experiments show that our 7B-parameter model achieves a 97% format validity rate, along with high pass rate, node-level and graph-level F1 scores, surpassing prior state-of-the-art methods that employ leading closed-source models such as GPT-4o and Claude series.Qualitative analysis further highlights our model's strength in synthesizing intricate workflows with diverse nodes, aligning with human instructions, and generalizing to newly introduced nodes, underscoring the potential of long CoT reasoning in AI art creation 1 .Node Selection: I will choose the nodes ['CLIPTextEncode', 'ControlNetLoader', ...].Workflow Planning: 1.The workflow begins by loading an InstantID model (`InstantIDModelLoader`) and preparing a face analysis model (`InstantIDFaceAnalysis`); 2. The reference image is loaded via the `LoadImage` node, and its facial keypoints are extracted with `FaceKeypointsPreprocessor`; 3. The conditioned model undergoes sampling Design a workflow that creates a high-resolution image of a woman while preserving the facial features of the reference image.The workflow should analyze the facial structure of t h e r e f e r e n c e i m a g e a n d integrate it into the generation process, ensuring the output aligns with the identity and facial features of the reference.
Authors: Zhenran Xu, Yiyu Wang, Yangxue, Longyue Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang
Institutions: Harbin Institute of Technology, Alibaba Group (China)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G2183668866, G2672035888
OpenAlex ID: W4417260954