Multi-modal fusion of in-situ video data and process parameters for online forecasting of cookie drying readiness
Year: 2026
Type: article
Abstract: Food drying is essential for food production, extending shelf life, and reducing transportation costs. Accurate real-time prediction of drying readiness is crucial for minimizing energy consumption, improving productivity, and ensuring product quality. However, real-time readiness prediction remains challenging because drying is a dynamic process, while existing methods often rely on offline measurements, manually engineered features, or single-modality process data. These limitations reduce their ability to provide sample-level online readiness forecasting during an active drying cycle. To address this gap, we propose a new end-to-end multi-modal data fusion framework that integrates in-situ video data with process parameters for real-time drying readiness forecasting. The proposed framework consists of a novel encoder–decoder architecture with modality-specific encoders and a transformer-based decoder, which can automatically and effectively extract key features from each modality. We apply the proposed approach to sugar cookie drying, where time-to-ready is predicted at each timestamp. Experimental results under leave-one-group-out cross-validation demonstrate that the proposed model achieves low prediction error and improves forecasting accuracy compared with both a conventional process-parameter baseline and a video-only ablation model. Additionally, the framework balances prediction accuracy, model size, and computational efficiency, making it a scalable solution for industrial drying. The proposed framework is extensible to various other industrial multi-modal fusion tasks for online decision-making.
Source: Journal of Food Engineering
Authors: Shichen Li(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing), Hao Feng(Funding acquisition, Methodology, Project administration, Writing – review & editing), Chenhui Shao(Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Visualization, Writing – original draft, Writing – review & editing)
Language: English
Topic: Food Drying and Modeling
Subfield: Food Science
Domain: Life Sciences
Open Access status: hybrid
Estimated APC paid: $4,350
Awards: G1453181110, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, G3562386289, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), NOVEL ENERGY-EFFICIENT DRYING TECHNOLOGIES FOR FOOD, PULP AND PAPER, AND OTHER ENERGY INTENSIVE MANUFACTURING INDUSTRIES, G5367127336, G5464919783, Category I: Bridging the Gap Between AI/ML Computing Demands and Today's Capabilities, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI
OpenAlex ID: W4414631726
