Work

Employee Work Behavior Monitoring Using Multimodal Large Language Models

Year: 2025

Type: article

Abstract: With the rapid advancement of artificial intelligence, enterprises increasingly demand efficient and flexible solutions for employee work behavior monitoring in office environments.Traditional systems often involve high costs, rigidity, and reliance on extensive labeled data.Multimodal large language models (MLLMs), capable of integrating information from text, images, and audio, offer a novel zero-shot inference approach that reduces data dependence and deployment complexity.In this study, we present a practical application framework combining seating area definition, image cropping, and prompt engineering to analyze employee behaviors such as focused screen engagement and nonwork-related interactions.Results are output in a standardized JavaScript Object Notation format facilitating aggregation and actionable insights for human resource management.Additionally, critical privacy and ethical and legal considerations are discussed, along with mitigation strategies to support responsible deployment.Through practical simulation scenarios and cost-benefit analysis, we demonstrate that MLLMs enable scalable and economically viable employee behavior monitoring solutions suitable for small and medium-sized enterprises.

Source: Sensors and Materials

DOI: 10.18494/sam5937

Authors: Yushi Chen, Chung-Hsing Chao, Linjing Liu, Cheng-Fu Yang

Institutions: Minghsin University of Science and Technology, Chaoyang University of Technology, National University of Kaohsiung

Language: English

Topic: AI and HR Technologies

Subfield: Organizational Behavior and Human Resource Management

Field: Business, Management and Accounting

Domain: Social Sciences

SDG: Decent work and economic growth

Open Access status: gold

Estimated APC paid: $748