Work

Multi-modal fusion of in-situ video data and process parameters for online forecasting of cookie drying readiness

Year: 2026
Type: article
Abstract: Food drying is essential for food production, extending shelf life, and reducing transportation costs. Accurate real-time prediction of drying readiness is crucial for minimizing energy consumption, improving productivity, and ensuring product quality. However, real-time readiness prediction remains challenging because drying is a dynamic process, while existing methods often rely on offline measurements, manually engineered features, or single-modality process data. These limitations reduce their ability to provide sample-level online readiness forecasting during an active drying cycle. To address this gap, we propose a new end-to-end multi-modal data fusion framework that integrates in-situ video data with process parameters for real-time drying readiness forecasting. The proposed framework consists of a novel encoder–decoder architecture with modality-specific encoders and a transformer-based decoder, which can automatically and effectively extract key features from each modality. We apply the proposed approach to sugar cookie drying, where time-to-ready is predicted at each timestamp. Experimental results under leave-one-group-out cross-validation demonstrate that the proposed model achieves low prediction error and improves forecasting accuracy compared with both a conventional process-parameter baseline and a video-only ablation model. Additionally, the framework balances prediction accuracy, model size, and computational efficiency, making it a scalable solution for industrial drying. The proposed framework is extensible to various other industrial multi-modal fusion tasks for online decision-making.
Language: English
Subfield: Food Science
Open Access status: hybrid
Estimated APC paid: $4,350
OpenAlex ID: W4414631726
FWCI: 1.89
Cites: 63
Cited by: 4
Related to: 10