Food Consumption Detection Through Hand Movements Using Smartwatch Sensors and Deep Learning Approaches

Abstract

Tracking food intake is crucial for maintaining well-being and managing various medical conditions. Traditional methods like manual records or food journals are often cumbersome and error-prone. This research introduces a novel method for identifying food intake by analyzing hand motions recorded by wristwatch sensors using deep learning algorithms. Data was collected from 51 participants wearing smartwatches with accelerometer and gyroscope sensors during eating and non-eating activities. The raw sensor data was pre-processed and segmented into defined-length windows, from which significant features were extracted. Several advanced deep learning models, including CNN and a hybrid CNN-ResBiLSTM model, were developed and trained to classify the sensor data into food consumption and nonconsumption activities. The models were evaluated using accuracy, loss, and F1-score metrics with 5-fold cross-validation. The CNN-ResBiLSTM model achieved the highest accuracy, 96.67% without and 98.05% with data augmentation using SMOTE, outperforming other baseline models. These results suggest that wristwatch sensors combined with deep learning algorithms offer a discreet and efficient method for tracking food intake, which could aid in monitoring and improving eating habits for various health purposes. This study contributes to the growing field of automated dietary monitoring, highlighting the potential of wearable technology and AI in promoting healthy eating behaviors.