Work

Facial Tracking Algorithms for Medication Intake Verification: A Scoping Review

Year: 2026

Type: article

Abstract: Background: Medication non-adherence is a major driver of poor therapeutic outcomes, and computer vision methods that observe facial movements offer a non-contact route to verifying oral medication intake. Objective: To map and synthesize the existing literature on computer vision techniques applicable to the monitoring of medication intake, focusing on face tracking methods, oral movement detection and deglutition recognition, and to assess their potential in supporting automatic medication adherence verification systems. Eligibility criteria: Peer-reviewed articles, conference papers, patents, theses and preprints published from 2016 onward, written in English or Portuguese, applying facial landmark tracking or face analysis to ingestion-related movements (mouth opening, hand-to-mouth motion, pill placement, mastication or deglutition); studies confined to object/pill detection without facial analysis, or to general food intake without transferability to medication, were excluded. Sources of evidence: A systematic screening of 362 initial records was conducted across six main electronic databases and repositories: Google Scholar, PubMed, ScienceDirect, arXiv, IEEE Xplore, and Espacenet. Charting methods: Data were charted with a standardized, pilot-tested extraction form capturing bibliographic attributes, dataset type, experimental environment, face tracking approach, tools/models, and target movements; extraction was performed by a single reviewer. Following the screening process, a final selection of 34 relevant studies was included for detailed analysis and mapping. Results: Among the 34 included studies, 14 employ facial landmarks, 11 utilize temporal deep learning models, 6 apply facial action models and 3 rely on hybrid multimodal approaches that combine video analysis, object detection and temporal modeling. Tasks such as detecting mouth opening or tracking pill-to-mouth movement show promising results, while accurately detecting deglutition remains a technical challenge due to high sensitivity and individual variability. Limitations: The majority of the literature relies on private or institutional datasets (31 studies) and operates in controlled laboratory environments (22 studies); only 2 studies evaluated their methods via independent external datasets, which limits the generalization of current solutions to real-world telemonitoring scenarios. Conclusions: The literature indicates the existence of solid technical foundations for developing automated medication intake verification systems. To advance the field toward practical deployment, future research must address the need for more diverse datasets, real-world validation and more robust, adaptable modeling frameworks.

Source: Applied Sciences

DOI: 10.3390/app16178453

Authors: Ruben Baptista, Fernanda Coutinho, João Quintas

Institutions: Polytechnic Institute of Coimbra, Instituto Pedro Nunes

Language: English

Topic: Face recognition and analysis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,896