Efficient Human Activity Recognition on Smartwatch Sensors Using Knowledge Distillation

Abstract

This study proposes a computationally efficient human activity recognition (HAR) framework on smartwatches by leveraging knowledge distillation techniques. It addresses the challenge of deploying deep learning models on resource-constrained wearable devices through the use of the WISDM-HARB dataset, which contains accelerometer and gyroscope data collected from both smartphones and smartwatches across 18 human activities. This approach transfers knowledge from a sophisticated teacher network comprising 12,648,664 parameters to a lightweight student model containing only 66,002 parameters. Despite this significant reduction in model size and complexity, the student network achieves 94.98% classification accuracy—closely matching the teacher model’s 95.14%—while reducing computational overhead by 98.48%, from 411.8 million to 6.2 million FLOPs, and model size by 99%. Extensive experiments across various hyperparameters show that lower values of the distillation coefficient (α = 0.1–0.2) and moderate temperature settings (T = 2–5) yield the best performance. These findings demonstrate that knowledge distillation can effectively compress deep HAR models without significant loss in accuracy, offering a practical solution for real-time activity recognition on low-power smartwatch devices.