Leveraging Fusion Methods of Human Pose and Motion Dynamics for Accurate Violence Detection in Video Surveillance

Abstract

The growing demand for security is leading to an increase in the installation of surveillance cameras and there is an urgent need for automated systems. Despite large number number of surveillance cameras are installed to detect and prevent violent activities, the need for monitoring and analyzing the footage in real time is a challenge. To accurately detect violence, it is essential to consider human interactions and minimize noise in surveillance footage. This study explores violence detection using a dual-stream deep neural network with human skeletons and motion changes as inputs extracted from surveillance videos. Furthermore, a weighted fusion strategy is employed, which integrates fusion operation functions, activation functions, and weighted outputs to prioritize the most relevant features from multiple inputs for effective violence detection. Our proposed framework is both effective and accurate, achieving accuracy of 95.05%. The use of skeletal data with a black background significantly improved performance compared to traditional RGB frames, while the incorporation of motion information and fusion strategy improved the performance of violence detection.