Human Suspicious Activity Detection in Video Using a Hybrid 3D CNN-LSTM

Abstract

Video surveillance is critical to public safety, watching several camera feeds requires a lot of manual effort and is susceptible to human error. The proposed work of this study is a hybrid model comprising 3D Conventional Neural Network (CNN) and Long Short Term Memory (LSTM) network that works as a combined system to learn both spatial and temporal features to automatically detect the violence in CCTV videos. The Real-Life Violence Situations dataset with 2,000 video clips was used in the experiments, which are publicly available. Five frames from each of the five videos were resized to 64 × 64 pixels and used for training the proposed model. The LSTM is to capture the long-term motion features, and the 3D-CNN to extract the spatial features and short-term motion features from the video, then to improve the recognition of violence. The proposed model is verified in terms of accuracy, precision, recall and F1-score and compared to a basic 2D CNN and MobileNetV3 model, conducted under the same experimental conditions. The proposed method outperformed MobileNetV3 86.0% and basic 2D CNN 87.0% in terms of accuracy 91.0%, precision 89.1%, recall 93.5% and F1-score 91.0%. The results show that the proposed approach of spatial temporal feature learning greatly outperforms frame-based approaches in terms of performance of detection of violence. A hybrid 3D CNN and LSTM model can detect violence in video with high accuracy. The design captures both spatial and temporal cues. Real-time deployment and larger datasets are needed for practical use.