Hierarchical Human Activity Recognition Based on Smartwatch Sensors Using Branch Convolutional Neural Networks

Abstract

Human activity recognition (HAR) has become a popular research topic in artificial intelligence thanks to the development of smart wearable devices. The main goal of human activity recognition is to efficiently recognize human behavior based on available data sources such as videos and images, including sensory data from wearable devices. Recently, HAR research has achieved promising results using learning-based approaches, especially deep learning methods. However, the need for high performance is still an open problem for researchers proposing new methods. In this work, we investigated the improvement of HAR by hierarchical classification based on smartwatch sensors using deep learning (DL) methods. To achieve the research goal, we introduced branch convolutional neural networks (B-CNNs) to accurately recognize human activities hierarchically and compared them with baseline models. To evaluate the deep learning models, we used a complex HAR benchmark dataset called WISDM-HARB dataset that collects smartwatch sensor data from 18 physical activities. The experimental results showed that the B-CNNs outperformed the baseline convolutional neural network (CNN) models when the hierarchical connection between classes was not considered. Moreover, the results confirmed that branch CNNs with class hierarchy improved the recognition performance with the highest accuracy of 95.84%.