Prompt‐Conditioned Cue Matching for Black‐Box Procedural Material Fitting
Year: 2026
Type: article
Abstract: Many procedural materials are implemented in legacy or production renderers where differentiable optimization is difficult to integrate, making black‐box parameter fitting still relevant. In this setting, the optimizer receives only a scalar feedback value, so the objective function determines which visual cues are observable to the search. Contrastive Language‐Image Pre‐training (CLIP) image matching is a strong baseline, but its global comparison can still underweight residual cues such as material color, feature scale, stripe spacing, gloss, or cellular structure. We propose Prompt‐Conditioned Cue Matching, a target‐relative CLIP response loss that uses a text prompt as a comparison axis between the target and candidate render. The prompt does not replace the target image; it specifies which cue should be compared more carefully. Because cue‐only optimization is weakly anchored, we combine this term with CLIP image matching in an image‐dominant hybrid loss. Experiments on checker, noise, wave, and Voronoi shaders, together with a real‐capture verification, show that language‐defined cue responses can provide targeted refinement when a visible residual cue remains.
Source: Computer Graphics Forum
DOI: 10.1111/cgf.70632
Authors: Runzhou Mao, Christoph Garth, Juraj Fulir
Language: English
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,710
OpenAlex ID: W7226008801
