Work

Development and Evaluation of Vision-Language Models for Ergonomic Physical Exposure Assessment in Occupational Manual Material Handling

Year: 2026
Type: dissertation
Abstract: Work-related musculoskeletal disorders remain a leading cause of lost workdays and substantial costs for workers and employers, and manual material handling (MMH) tasks are a major contributor to the physical exposures that drive this risk. Existing physical exposure assessment methods, including self-reports, systematic observation, and direct measurement, are often labor-intensive, costly, or impractical for use in real work environments. Vision-language models (VLMs), which combine visual encoders with large language models to jointly process video and text, offer a potentially non-invasive and scalable alternative for quantifying physical exposures from regular RGB video. This dissertation developed and evaluated VLM-based methods for physical exposure assessment during MMH tasks using three sequential studies. The first study evaluated the use of VLMs to classify eight MMH tasks and specific task conditions (hand configuration and lift origin), and to temporally segment task start and end times. The work achieved task classification accuracies of ~82–85%, lift-origin classification accuracies of ~94–98%, and errors in mean absolute start and end time of <0.5 seconds. The second study assessed the feasibility of using two VLM-based pipelines to estimate horizontal (H) and vertical (V) hand distances required by the Revised NIOSH Lifting Equation (RNLE). Using a segmentation-based, multi-view pipeline consistently yielding the smallest errors (mean absolute errors of ~6–8 cm for H and ~5–8 cm for V). The third study evaluated a VLM-based pipeline for estimating dynamic, triaxial, bilateral external hand forces during MMH tasks. Root mean square errors of ~4.7–5.6 N were obtained for the horizontal and mediolateral force components and ~10.6–11.0 N for the vertical component. Collectively, these findings demonstrate the feasibility of VLM-based methods as objective, scalable, and minimally intrusive tools for quantifying physical exposures during MMH tasks, supporting their continued development toward practical, video-based occupational physical exposure assessment.
Institution: Virginia Tech
Language: English
Open Access status: green
OpenAlex ID: W7221837316
Cites: 217
Cited by:
Related to: 10