BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis
Year: 2026
Type: article
Abstract: This paper presents BodySync AI, a multimodal calorie-tracking prototype that combines optical character recognition (OCR), object detection, pose estimation, and an agentic large language model (LLM). The system supports two tasks: extracting nutritional values from photographs of food-packaging labels, and analyzing body proportions from full-body images. Nutrition extraction uses a classical image-processing front end (perspective correction, denoising, illumination normalization, and contrast enhancement) followed by EasyOCR and rulebased parsing of calories, fat, carbohydrates, and protein. In parallel, we train a YOLOv8n nutrition-label detector on the Open Food Facts Nutrition Table Detection dataset and study the effect of training epochs, learning rate, optimizer, batch size, and input resolution in 14 controlled runs. The best configuration reaches an mAP@50 of 0.904 and an mAP@50-95 of 0.727 on the validation split. Body analysis uses MediaPipe Pose landmarks to compute normalized shoulder width, hip width, torso length, and shoulder-to-hip ratio, which feed a baseline rule-based bodytype estimate. A Groq-hosted Llama 3.1 8B agent exposes these components as tools through a natural-language interface. We describe the system as built, report the detector experiments, and outline the evaluations and learned body-composition models planned as future work.
Institution: San Diego State University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
SDG: Zero hunger
Open Access status: hybrid
OpenAlex ID: W7234428606
