A Review Work: Human Action Recognition in Video Surveillance Using Deep Learning Techniques

Nukala Sujata Gupta; K. Ruth Ramya; Ramesh Karnati

doi:10.15622/ia.23.2.5

Нукала Суджата Гупта Образовательный фонд Конеру Лакшмайи
К. Рут Рамья Образовательный фонд Конеру Лакшмайи
Рамеш Карнати Инженерный колледж Вардхамана

DOI:

https://doi.org/10.15622/ia.23.2.5

Ключевые слова:

распознавание лиц, распознавание эмоций, распознавание действий, распознавание аномалий, DL, распознавание человеческого поведения и активности /обнаружение

Аннотация

Несмотря на широкое применение во многих областях, точная и эффективная идентификация деятельности человека продолжает оставаться интересной исследовательской проблемой в области компьютерного зрения. В настоящее время проводится много исследований по таким темам, как распознавание активности пешеходов и способы распознавания движений людей с использованием данных глубины, трехмерных скелетных данных, данных неподвижных изображений или стратегий, использующих пространственно-временные точки интереса. Это исследование направлено на изучение и оценку подходов DL для обнаружения человеческой активности на видео. Основное внимание было уделено нескольким структурам для обнаружения действий человека, которые используют DL в качестве своей основной стратегии. В зависимости от приложения, включая идентификацию лиц, идентификацию эмоций, идентификацию действий и идентификацию аномалий, прогнозы появления людей разделены на четыре различные подкатегории. В литературе было проведено несколько исследований, основанных на этих распознаваниях для прогнозирования поведения и активности человека в приложениях видеонаблюдения. Сравнивается современное состояние методов DL для четырех различных приложений. В этой статье также представлены области применения, научные проблемы и потенциальные цели в области распознавания человеческого поведения и активности на основе DL.

Литература

1. Zhang J., Zi L., Hou Y., Wang M., Jiang W., Deng D. A DL-based approach to enable action recognition for construction equipment. Advances in Civil Engineering. 2020. pp. 1–14.
2. Wang X., Che Z., Jiang B., Xiao N., Yang K., Tang J., Ye J., Wang J., Qi Q. Robust unsupervised video anomaly detection by multipath frame prediction. IEEE transactions on neural networks and learning systems. 2021. vol. 33. no. 6. pp. 2301–2312.
3. Zhang H.B., Zhang Y.X., Zhong B., Lei Q., Yang L., Du J.X., Chen D.S. A comprehensive survey of vision-based human action recognition methods. Sensors. 2019. vol. 19(5). no. 1005.
4. Pervaiz M., Jalal A., Kim K. A hybrid algorithm for multi-people counting and tracking for smart surveillance. International Bhurban conference on applied sciences and technologies (IBCAST). 2021. pp. 530–535.
5. Kong Y., Fu Y. Human action recognition and prediction: A survey. International Journal of Computer Vision. 2022. vol. 130(5). pp. 1366–1401.
6. Franco A., Magnani A., Maio D. A multimodal approach for human activity recognition based on skeleton and RGB data. Pattern Recognition Letters. 2020. vol. 131. pp. 293–299.
7. Wang L., Huynh D.Q., Koniusz P. A comparative review of recent kinect-based action recognition algorithms. IEEE Transactions on Image Processing. 2019. vol. 29. pp. 15–28.
8. Zhou X., Liang W., Kevin I., Wang K., Wang H., Yang L.T., Jin Q. Deep-learning-enhanced human activity recognition for the Internet of Healthcare things. IEEE Internet of Things Journal. 2020. vol. 7(7). pp. 6429–6438.
9. Qiu Z., Yao T., Ngo C.W., Tian X., Mei T. Learning spatio-temporal representation with local and global diffusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. pp. 12056–12065.
10. Sreenu G., Durai S. Intelligent video surveillance: a review through DL techniques for crowd analysis. Journal of Big Data. 2019. vol. 6(1). pp. 1–27.
11. Elharrouss O., Almaadeed N., Al-Maadeed S., Bouridane A., Beghdadi A. A combined multiple action recognition and summarization for surveillance video sequences. Applied Intelligence. 2021. vol. 51. pp. 690–712.
12. Jaouedi N., Boujnah N., Bouhlel M.S. A new hybrid DL model for human action recognition. Journal of King Saud University – Computer and Information Sciences. 2020. vol. 32. no. 4. pp. 447–453.
13. Dang L.M., Min K., Wang H., Piran M.J., Lee C.H., Moon H. Sensor-based and vision-based human activity recognition: A comprehensive survey. Pattern Recognition. 2020. vol. 108. no. 107561.
14. Saeed A., Ozcelebi T., Lukkien J. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2019. vol. 3(2). pp. 1–30.
15. Fu B., Damer N., Kirchbuchner F., Kuijper A. Sensing technology for human activity recognition: A comprehensive survey. IEEE Access. 2020. vol. 8. pp. 83791–83820.
16. du Toit J., du Toit T, Kruger H. Heuristic Data Augmentation for Improved Human Activity Recognition. Proceedings of the Southern Africa Telecommunication Networks and Applications Conference (SATNAC). 2019. pp. 264–269.
17. Rezaee K., Rezakhani S.M., Khosravi M.R., Moghimi M.K. A survey on DL-based real-time crowd anomaly detection for secure distributed video surveillance. Personal and Ubiquitous Computing. 2021. pp. 1–17.
18. Concone F., Re G.L., Morana M. A fog-based application for human activity recognition using personal smart devices. ACM Transactions on Internet Technology (TOIT). 2019. vol. 19(2). pp. 1–20.
19. He J.Y., Wu X., Cheng Z.Q., Yuan Z., Jiang Y.G. DB-LSTM: Densely-connected Bi-directional LSTM for human action recognition. Neurocomputing. 2021. vol. 444. pp. 319–331.
20. Beddiar D.R., Nini B., Sabokrou M., Hadid A. Vision-based human activity recognition: a survey. Multimedia Tools and Applications. 2020. vol. 79. no. 41-42. pp. 30509–30555.
21. Chen J., Li K., Deng Q., Li K., Philip S.Y. Distributed DL model for intelligent video surveillance systems with edge computing. IEEE Transactions on Industrial Informatics. 2019. DOI: 10.1109/TII.2019.2909473.
22. Zhao Y., Shen X., Jin Z., Lu H., Hua X.S. Attribute-driven feature disentangling and temporal aggregation for video person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. pp. 4913–4922.
23. Kaur G., Sinha R., Tiwari P.K., Yadav S.K., Pandey P., Raj R., Vashisth A., Rakhra M. Face mask recognition system using CNN model. Neuroscience Informatics. 2021. vol. 2(3). no. 100035. DOI:10.1016/j.neuri.2021.100035.
24. Wang Y., Yue Y., Lin Y., Jiang H., Lai Z., Kulikov V., Huang G. Adafocus v2: End-to-end training of spatial dynamic networks for video recognition. IEEE/CVF Computer Vision and Pattern Recognition (CVPR). 2022. pp. 20030–20040.
25. Goyal H., Sidana K., Singh C., Jain A., Jindal S. A real-time face mask detection system using a convolutional neural network. Multimedia Tools and Applications. 2022. vol. 81(11). pp. 14999–15015.
26. Sayeed A., Srizon A.Y., Hasan M.M., Shin J., Hasan M.A.M., Mahmud M.R. A Hybrid Campus Security System Combined Face, Number-Plate, and Voice Recognition. International Conference on Recent Trends in Image Processing and Pattern Recognition. 2022. pp. 356–368.
27. Kumar B.A., Bansal M. Face Mask Detection on Photo and Real-Time Video Images Using Caffe-MobileNetV2 Transfer Learning. Applied Sciences. 2023. vol. 13(2). no. 935.
28. Kamyab T., Daealhaq H., Ghahfarokhi A.M., Beheshtinejad F., Salajegheh E. Combination of Genetic Algorithm and Neural Network to Select Facial Features in Face Recognition Technique. International Journal of Robotics and Control Systems. 2023. vol. 3(1). pp. 50–58.
29. Singh A., Bhatt S., Nayak V., Shah M. Automation of surveillance systems using DL and facial recognition. International Journal of System Assurance Engineering and Management. 2023. vol. 14. pp. 236–245.
30. Terhorst P., Ihlefeld M., Huber M., Damer N., Kirchbuchner F., Raja K., Kuijper A. Qmagface: Simple and accurate quality-aware face recognition. In Proceedings of the IEEE/CVF Applications of Computer Vision. 2023. 3484–3494.
31. Wang K., Peng X., Yang J., Meng D., Qiao Y. Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing. 2020. vol. 29. pp. 4057–4069.
32. Hossain M.S., Muhammad G. Emotion recognition using DL approach from audio–visual emotional big data. Information Fusion. 2019. vol. 49. pp. 69–78.
33. Kanjo E., Younis E.M., Ang C.S. DL analysis of mobile physiological, environmental, and location sensor data for emotion detection. Information Fusion. 2019. vol. 49. pp. 46–56.
34. Wang K., Peng X., Yang J., Lu S., Qiao Y. Suppressing uncertainties for large-scale facial expression recognition. Proceedings of the IEEE/CVF computer vision and pattern recognition. 2020. pp. 6897–6906.
35. Minaee S., Minaei, M., Abdolrashidi A. Deep-emotion: Facial expression recognition using the attentional convolutional network. Sensors. 2021. vol. 21(9). no. 3046.
36. Umer S., Rout R.K., Pero C., Nappi M. Facial expression recognition with trade-offs between data augmentation and DL features. Journal of Ambient Intelligence and Humanized Computing. 2022. pp. 1–15.
37. Zhang W., Qiu F., Wang S., Zeng H., Zhang Z., An R., Ma B., Ding Y. Transformer-based multimodal information fusion for facial expression analysis. Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition. 2022. pp. 2428–2437.
38. Zhu X., Li Z., Sun J. Expression recognition method combining convolutional features and Transformer. Mathematical Foundations of Computing. 2023. vol. 6. no. 2. pp. 203–217.
39. Bapat M.M., Patil C.H., Mali S.M. Database Development and Recognition of Facial Expression using DL. 2023. 20 p. DOI: 10.21203/rs.3.rs-2477808/v1.
40. Mukhiddinov M., Djuraev O., Akhmedov F., Mukhamadiyev A., Cho J. Masked Face Emotion Recognition Based on Facial Landmarks and DL Approaches for Visually Impaired People. Sensors. 2023. vol. 23(3). no. 1080.
41. Xia K., Huang J., Wang H. LSTM-CNN architecture for human activity recognition. IEEE Access. 2020. vol. 8. pp. 56855–56866.
42. Dhiman C., Vishwakarma D.K. View-invariant deep architecture for human action recognition using two-stream motion and shape temporal dynamics. IEEE Transactions on Image Processing. 2020. vol. 29. pp. 3835–3844.
43. Paoletti G., Cavazza J., Beyan C., Del Bue A. Unsupervised human action recognition with skeletal graph Laplacian and self-supervised viewpoints invariance. 2022. arXiv preprint arXiv:2204.10312.
44. Sanchez-Caballero A., de Lopez-Diz S., Fuentes-Jimenez D., Losada-Gutiérrez C., Marrón-Romera M., Casillas-Perez D., Sarker M.I. 3dfcnn: Real-time action recognition using 3d deep neural networks with raw depth information. Multimedia Tools and Applications. 2022. vol. 81. no. 17. pp. 24119–24143.
45. Khan I.U., Afzal S., Lee J.W. Human activity recognition via hybrid DL-based model. Sensors. 2022. vol. 22(1). no. 323.
46. Yadav S.K., Tiwari K., Pandey H.M., Akbar S.A. Skeleton-based human activity recognition using Conv LSTM and guided feature learning. Soft Computing. 2022. pp. 1–14.
47. Zhu Q., Deng H. Spatial adaptive graph convolutional network for skeleton-based action recognition. Applied Intelligence. 2023. pp. 1–13.
48. Singh G., Choutas V., Saha S., Yu F., Van Gool L. Spatio-Temporal Action Detection under Large Motion. Proceedings of the IEEE/CVF Applications of Computer Vision. 2023. pp. 6009–6018.
49. Ahn D., Kim S., Hong H., Ko B.C. STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition. In Proceedings of the IEEE/CVF Applications of Computer Vision. 2023. pp. 3330–3339.
50. Peng K., Roitberg A., Yang K., Zhang J., Stiefelhagen R. Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions. IEEE Transactions on Multimedia. 2023. arXiv preprint arXiv:2202.11423v3.
51. Zhou J.T., Du J., Zhu H., Peng X., Liu Y., Goh R.S.M. Anomalynet: An anomaly detection network for video surveillance. IEEE Transactions on Information Forensics and Security. 2019. vol. 14(10). pp. 2537–2550.
52. Pawar K., Attar V. DL-based detection and localization of road accidents from traffic surveillance videos. ICT Express. 2022. vol. 8. no. 3. pp. 379–387.
53. Ganokratanaa T., Aramvith S., Sebe N. Video anomaly detection using deep residual-spatiotemporal translation network. Pattern Recognition Letters. 2022. vol. 155. pp. 143–150.
54. Roa’a M., Aljazaery I.A., ALRikabi H.T.S., Alaidi A.H.M. Automated Cheating Detection Based on Video Surveillance in the Examination Classes. iJIM. 2022. vol. 16(08). no. 125.
55. Kamoona A.M., Gostar A.K., Bab-Hadiashar A., Hoseinnezhad R. Multiple instance-based video anomaly detection using deep temporal encoding–decoding. Expert Systems with Applications. 2023. vol. 214. no. 119079. DOI: 10.1016/j.eswa.2022.119079.
56. Le V.T., Kim Y.G. Attention-based residual autoencoder for video anomaly detection. Applied Intelligence. 2023. vol. 53(3). pp. 3240–3254.
57. Abbas Z.K., Al-Ani A.A. An adaptive algorithm based on principal component analysis-DL for anomalous events detection. Indonesian Journal of Electrical Engineering and Computer Science. 2023. vol. 29(1). pp. 421–430.
58. Pazho A.D., Neff C., Noghre G.A., Ardabili B.R., Yao S., Baharani M., Tabkhi H. Ancilia: Scalable Intelligent Video Surveillance for the Artificial Intelligence of Things. 2023. arXiv preprint arXiv:2301.03561.
59. Thakare K.V., Raghuwanshi Y., Dogra D.P., Choi H., Kim I.J. DyAnNet: A Scene Dynamicity Guided Self-Trained Video Anomaly Detection Network. Proceedings of the IEEE/CVF Applications of Computer Vision. 2023. pp. 5541–5550.
60. Deng H., Zhang Z., Zou S., Li X. Bi-Directional Frame Interpolation for Unsupervised Video Anomaly Detection. In Proceedings of the IEEE/CVF Applications of Computer Vision. 2023. pp. 2634–2643.

Просмотры	709
Скачивания	330

Искусственный интеллект, инженерия данных и знаний

Распознавание действий человека в системах видеонаблюдения с использованием методов глубокого обучения – обзор

DOI:

Ключевые слова:

Аннотация

Литература

Опубликован

Статистика

Как цитировать

Выпуск

Раздел

Импакт-фактор

Разделы

Мы в сети

Обратная связь