International Journal of Nonlinear Analysis and Applications
A survey of deep learning-based object detection: Application and open issues
Document Type : Review articles
Authors
Department of computer science, College of science, University of Diyala, Baqubah, Iraq
Abstract
Object tracking and detection are among the most significant jobs in computer vision, having many applications in areas, which includes autonomous vehicle tracking, robotics, as well as traffic monitoring. Several studies have been conducted in past years. However, since detecting various problems, for instance, fast motion, illumination variations, as well as occlusion, study in this field persists. Furthermore, deep convolutional neural networks (DCNNs) have grown increasingly significant for object detection as deep learning (DL) techniques have advanced. As a result, numerous approaches for object detection are studied in this research, as well as a comprehensive. This project encompasses backbone networks, loss functions and training strategies, classical object detection architectures, complex problems, datasets and evaluation metrics, applications, future development directions, as well as a review and analysis of DL-based object detection techniques conducted in previous years. Experts in the field of object detection will benefit from this review article.
Keywords
[1] I. Ahmed and G. Jeon, A real-time person tracking system based on SiamMask network for intelligent video
surveillance, J. Real-Time Image Process. 18 (2021), no. 5, 1803–1814.
[2] A.F. Al-Battal, Y. Gong, L. Xu, T. Morton, C. Du, Y. Bu, I.R. Lerman, R. Madhavan and T.Q. Nguyen, A CNN
segmentation-based approach to object detection and tracking in ultrasound scans with application to the vagus
nerve detection, Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, 2021, pp. 3322–3327.
[3] D. Anitta, Human head pose estimation based on HF method, Microprocess. Microsyst. 82 (2021), 103802.
[4] M.R. Bachute and J.M. Subhedar, Autonomous driving architectures: insights of machine learning and deep
learning algorithms, Mach. Learn. Appl. 6 (2021), 100164.
[5] L. Bertinetto, J. Valmadre, J.F. Henriques, A. Vedaldi and P.H.S. Torr, Fully-convolutional siamese networks for
object tracking, Eur. Conf. Comput. Vis. Springer, Cham., 2016, pp. 850–865.
[6] A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, Yolov4: optimal speed and accuracy of object detection, arXiv
Prepr. arXiv2004.10934, (2020).
[7] J. Chen, C. Zhang, J. Luo, J. Xie and Y. Wan, Driving maneuvers prediction based autonomous driving control
by deep monte carlo tree search, IEEE Trans. Veh. Technol. 69 (2020), no. 7, 7146–7158.
[8] H.-K. Chiu, J. Li, R. Ambrus and J. Bohg, Probabilistic 3d multi-modal, multi-object tracking for autonomous
driving, IEEE Int. Conf. Robotics and Automation (ICRA), 2021, pp. 14227–14233.[9] G. Ciaparrone, F. Luque S´anchez, S. Tabik, L. Troiano, R. Tagliaferri and F. Herrera, Deep learning in video
multi-object tracking: a survey, Neurocomput. 381 (2020), 61–88.
[10] Y. Cui, R. Chen, W. Chu, L. Chen, D. Tian, Y. Li and D. Cao, Deep learning for image and point cloud fusion
in autonomous driving: a review, IEEE Trans. Intell. Transp. Syst. 23 (2022), no. 2, 722–739.
[11] Y. Deng, T. Zhang, G. Lou, X. Zheng, J. Jin and Q.L. Han, Deep learning-based autonomous driving systems: a
survey of attacks and defenses, IEEE Trans. Ind. Inf. 17 (2021), no. 12, 7897–7912.
[12] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn and A. Zisserman, The pascal visual object classes (voc)
challenge, Int. J. Comput. Vis. 88 (2010), no. 2, 303–338.
[13] H. Fujiyoshi, T. Hirakawa and T. Yamashita, Deep learning-based image recognition for autonomous driving,
IATSS Res. 43 (2019), no. 4, 244–252.
[14] K. He, G. Gkioxari, P. Doll and R. Girshick, Mask r-cnn, Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 2961–
2969.
[15] N. Ijaz and Y. Wang, Automatic steering angle and direction prediction for autonomous driving using deep learning, Proc. Int. Symp. Comput. Sci. Intell. Control. ISCSIC 2021, pp. 280–283.
[16] L. Kalake, W. Wan and L. Hou, Analysis based on recent deep learning approaches applied in real-time multi-object
tracking: a review, IEEE Access 9 (2021), 32650–32671.
[17] A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci and
T. Duerig, The open images dataset V4: unified image classification, object detection, and visual relationship
detection at scale, arXiv 2018. arXiv preprint arXiv:1811.00982, (2018).
[18] G. Li, Y. Yang, X. Qu, D. Cao and K. Li, A deep learning based image enhancement approach for autonomous
driving at night, Knowledge-Based Syst. 213 (2021), 106617.
[19] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ar and C.L. Zitnick, Microsoft coco:
common objects in context, Eur. Conf. Comput. Vision, 2014, pp. 740–755.
[20] G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, J.A. Van Der Laa, B. Van Ginneken
and C.I. S´anchez, A survey on deep learning in medical image analysis, Med. Image Anal. 42 (2017), 60–88.
[21] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu and A.C. Berg, Ssd: single shot multibox detector,
Eur. Conf. Comput. Vision, 2016, pp. 21–37.
[22] Y. Liu, P. Sun, N. Wergeles and Y. Shang, A survey and performance evaluation of deep learning methods for
small object detection, Expert Syst. Appl. 172 (2021), 114602.
[23] C. Liu, Y. Tao, J. Liang, K. Li and Y. Chen, Object detection based on YOLO network, Proc. 2018 IEEE 4th Inf.
Technol. Mechatronics Eng. Conf. ITOEC 2018, pp. 799–803.
[24] X. Ma, W. Ouyang, A. Simonelli and E. Ricci, 3d object detection from images for autonomous driving: a survey,
arXiv preprint arXiv:2202.02980, (2022), 1–26.
[25] A. Makandar, D. Mulimani and M. Jevoor, Preprocessing step–review of key frame extraction techniques for object
detection in video, Int. J. Curr. Eng. Technol. 5 (2015), no. 3, 2036–2039.
[26] K. Muhammad, A. Ullah, J. Lloret, J. Del Ser and V.H.C. De Albuquerque, Deep learning for safe autonomous
driving: current challenges and future directions, IEEE Trans. Intell. Transp. Syst. 22 (2021), no. 7, 4316–4336.
[27] M. Pervaiz, Y.Y. Ghadi, M. Gochoo, A. Jalal, S. Kamal and D.S. Kim, A smart surveillance system for people
counting and tracking using particle flow and modified som, Sustain. 13 (2021), no. 10, 1–20.
[28] G. Prabhakar, B. Kailath, S. Natarajan and R. Kumar, Obstacle detection and classification using deep learning
for tracking in high-speed autonomous driving, TENSYMP 2017 - IEEE Int. Symp. Technol. Smart Cities, 2017,
pp. 3–8.
[29] A. Raghunandan, P. Raghav and H.R. Aradhya, Object detection algorithms for video surveillance applications,
Proc. 2018 IEEE Int. Conf. Commun. Signal Process. ICCSP 2018, pp. 563–568.
[30] K. Ragland and P. Tharcis, A survey on object detection, classification and tracking methods, Int. J. Eng. Res.Technol. 3 (2014), no. 11, 622–628.
[31] J. Redmon and A. Farhadi, YOLO9000: better, faster, stronger, Proc. IEEE Int. Conf. Comput. Vis. Pattern
Recog., (2017), pp. 7263–7271.
[32] S. Ren, K. He, R. Girshick and J. Sun, Faster r-cnn: towards real-time object detection with region proposal
networks, Adv. Neural Inf. Process. Syst. 28 (2015).
[33] F. Rosique, P.J. Navarro, C. Fern´andez and A. Padilla, A systematic review of perception system and simulators
for autonomous vehicles research, Sensors 19 (2019), no. 3.
[34] L. Rupasinghe and M.C. Liyanapathirana, Human tracking and profiling for risk management, Global J. Comput.
Sci. Technol. 22 (2022), no. 1.
[35] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein
and A.C. Berg, Imagenet large scale visual recognition challenge, Int. J. Comput. Vis. 115 (2015), no. 3, 211–252.
[36] A. Shafique, G. Cao, Z. Khan, M. Asad and M. Aslam, Deep learning-based change detection in remote sensing
images: a review, Remote Sens. 14 (2022) , no. 4, 1–40.
[37] V. Sharma and R.N. Mir, A comprehensive and systematic look up into deep learning based object detection
techniques: a review, Comput. Sci. Rev. 38 (2020), 100301.
[38] K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 3rd Int. Conf.
Learn. Represent. ICLR 2015 - Conf. Track Proc. 2015, pp. 1–14.
[39] Z. Soleimanitaleb, M.A. Keyvanrad and A. Jafari, Object tracking methods: a review, 9th Int. Conf. Comput.
Knowl. Eng. ICCKE 2019, pp. 282–288.
[40] P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine and
V. Vasudevan, Scalability in perception for autonomous driving: waymo open dataset, Proc. IEEE Comput. Soc.
Conf. Comput. Vis. Pattern Recognit. 2020, pp. 2443–2451.
[41] A. U¸car, Y. Demir and C. G¨uzeli¸s, Object recognition and detection with deep learning for autonomous driving
applications, Simulation 93 (2017), no. 9, 759–769.
[42] K.E.A. Van De Sande, J.R.R. Uijlings, T. Gevers and A.W.M. Smeulders, Segmentation as selective search for
object recognition, Proc. IEEE Int. Conf. Comput. Vis. 2011, no. 2, pp. 1879–1886.
[43] M. Waheed, M. Javeed and A. Jalal, A novel deep learning model for understanding two-person interactions using
depth sensors, Int. Conf. Innov. Comput. (ICIC), IEEE, 2022, 1–8.
[44] T. Wollmann and K. Rohr, Deep consensus network: aggregating predictions to improve object detection in microscopy images, Med. Image Anal. 70 (2021), 102019.
[45] Y. Yin, Design of deep learning based autonomous driving control algorithm, 2nd Int. Conf. Consumer Electron.
Comput. Engin. (ICCECE), IEEE. 2022, pp. 423–426.
[46] Z. Zhang, Y. Li, W. Wu, H. Chen, L. Cheng and S. Wang, Tumor detection using deep learning method in
automated breast ultrasound, Biomed. Signal Process. Control 68 (2021), 102677.
[47] H.Y. Zhou, C. Wang, H. Li, G. Wang, S. Zhang, W. Li and Y. Yu, SSMD: semi-supervised medical image detection
with adaptive consistency and heterogeneous perturbation, Med. Image Anal. 72 (2021), 102117.
surveillance, J. Real-Time Image Process. 18 (2021), no. 5, 1803–1814.
[2] A.F. Al-Battal, Y. Gong, L. Xu, T. Morton, C. Du, Y. Bu, I.R. Lerman, R. Madhavan and T.Q. Nguyen, A CNN
segmentation-based approach to object detection and tracking in ultrasound scans with application to the vagus
nerve detection, Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, 2021, pp. 3322–3327.
[3] D. Anitta, Human head pose estimation based on HF method, Microprocess. Microsyst. 82 (2021), 103802.
[4] M.R. Bachute and J.M. Subhedar, Autonomous driving architectures: insights of machine learning and deep
learning algorithms, Mach. Learn. Appl. 6 (2021), 100164.
[5] L. Bertinetto, J. Valmadre, J.F. Henriques, A. Vedaldi and P.H.S. Torr, Fully-convolutional siamese networks for
object tracking, Eur. Conf. Comput. Vis. Springer, Cham., 2016, pp. 850–865.
[6] A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, Yolov4: optimal speed and accuracy of object detection, arXiv
Prepr. arXiv2004.10934, (2020).
[7] J. Chen, C. Zhang, J. Luo, J. Xie and Y. Wan, Driving maneuvers prediction based autonomous driving control
by deep monte carlo tree search, IEEE Trans. Veh. Technol. 69 (2020), no. 7, 7146–7158.
[8] H.-K. Chiu, J. Li, R. Ambrus and J. Bohg, Probabilistic 3d multi-modal, multi-object tracking for autonomous
driving, IEEE Int. Conf. Robotics and Automation (ICRA), 2021, pp. 14227–14233.[9] G. Ciaparrone, F. Luque S´anchez, S. Tabik, L. Troiano, R. Tagliaferri and F. Herrera, Deep learning in video
multi-object tracking: a survey, Neurocomput. 381 (2020), 61–88.
[10] Y. Cui, R. Chen, W. Chu, L. Chen, D. Tian, Y. Li and D. Cao, Deep learning for image and point cloud fusion
in autonomous driving: a review, IEEE Trans. Intell. Transp. Syst. 23 (2022), no. 2, 722–739.
[11] Y. Deng, T. Zhang, G. Lou, X. Zheng, J. Jin and Q.L. Han, Deep learning-based autonomous driving systems: a
survey of attacks and defenses, IEEE Trans. Ind. Inf. 17 (2021), no. 12, 7897–7912.
[12] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn and A. Zisserman, The pascal visual object classes (voc)
challenge, Int. J. Comput. Vis. 88 (2010), no. 2, 303–338.
[13] H. Fujiyoshi, T. Hirakawa and T. Yamashita, Deep learning-based image recognition for autonomous driving,
IATSS Res. 43 (2019), no. 4, 244–252.
[14] K. He, G. Gkioxari, P. Doll and R. Girshick, Mask r-cnn, Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 2961–
2969.
[15] N. Ijaz and Y. Wang, Automatic steering angle and direction prediction for autonomous driving using deep learning, Proc. Int. Symp. Comput. Sci. Intell. Control. ISCSIC 2021, pp. 280–283.
[16] L. Kalake, W. Wan and L. Hou, Analysis based on recent deep learning approaches applied in real-time multi-object
tracking: a review, IEEE Access 9 (2021), 32650–32671.
[17] A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci and
T. Duerig, The open images dataset V4: unified image classification, object detection, and visual relationship
detection at scale, arXiv 2018. arXiv preprint arXiv:1811.00982, (2018).
[18] G. Li, Y. Yang, X. Qu, D. Cao and K. Li, A deep learning based image enhancement approach for autonomous
driving at night, Knowledge-Based Syst. 213 (2021), 106617.
[19] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ar and C.L. Zitnick, Microsoft coco:
common objects in context, Eur. Conf. Comput. Vision, 2014, pp. 740–755.
[20] G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, J.A. Van Der Laa, B. Van Ginneken
and C.I. S´anchez, A survey on deep learning in medical image analysis, Med. Image Anal. 42 (2017), 60–88.
[21] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu and A.C. Berg, Ssd: single shot multibox detector,
Eur. Conf. Comput. Vision, 2016, pp. 21–37.
[22] Y. Liu, P. Sun, N. Wergeles and Y. Shang, A survey and performance evaluation of deep learning methods for
small object detection, Expert Syst. Appl. 172 (2021), 114602.
[23] C. Liu, Y. Tao, J. Liang, K. Li and Y. Chen, Object detection based on YOLO network, Proc. 2018 IEEE 4th Inf.
Technol. Mechatronics Eng. Conf. ITOEC 2018, pp. 799–803.
[24] X. Ma, W. Ouyang, A. Simonelli and E. Ricci, 3d object detection from images for autonomous driving: a survey,
arXiv preprint arXiv:2202.02980, (2022), 1–26.
[25] A. Makandar, D. Mulimani and M. Jevoor, Preprocessing step–review of key frame extraction techniques for object
detection in video, Int. J. Curr. Eng. Technol. 5 (2015), no. 3, 2036–2039.
[26] K. Muhammad, A. Ullah, J. Lloret, J. Del Ser and V.H.C. De Albuquerque, Deep learning for safe autonomous
driving: current challenges and future directions, IEEE Trans. Intell. Transp. Syst. 22 (2021), no. 7, 4316–4336.
[27] M. Pervaiz, Y.Y. Ghadi, M. Gochoo, A. Jalal, S. Kamal and D.S. Kim, A smart surveillance system for people
counting and tracking using particle flow and modified som, Sustain. 13 (2021), no. 10, 1–20.
[28] G. Prabhakar, B. Kailath, S. Natarajan and R. Kumar, Obstacle detection and classification using deep learning
for tracking in high-speed autonomous driving, TENSYMP 2017 - IEEE Int. Symp. Technol. Smart Cities, 2017,
pp. 3–8.
[29] A. Raghunandan, P. Raghav and H.R. Aradhya, Object detection algorithms for video surveillance applications,
Proc. 2018 IEEE Int. Conf. Commun. Signal Process. ICCSP 2018, pp. 563–568.
[30] K. Ragland and P. Tharcis, A survey on object detection, classification and tracking methods, Int. J. Eng. Res.Technol. 3 (2014), no. 11, 622–628.
[31] J. Redmon and A. Farhadi, YOLO9000: better, faster, stronger, Proc. IEEE Int. Conf. Comput. Vis. Pattern
Recog., (2017), pp. 7263–7271.
[32] S. Ren, K. He, R. Girshick and J. Sun, Faster r-cnn: towards real-time object detection with region proposal
networks, Adv. Neural Inf. Process. Syst. 28 (2015).
[33] F. Rosique, P.J. Navarro, C. Fern´andez and A. Padilla, A systematic review of perception system and simulators
for autonomous vehicles research, Sensors 19 (2019), no. 3.
[34] L. Rupasinghe and M.C. Liyanapathirana, Human tracking and profiling for risk management, Global J. Comput.
Sci. Technol. 22 (2022), no. 1.
[35] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein
and A.C. Berg, Imagenet large scale visual recognition challenge, Int. J. Comput. Vis. 115 (2015), no. 3, 211–252.
[36] A. Shafique, G. Cao, Z. Khan, M. Asad and M. Aslam, Deep learning-based change detection in remote sensing
images: a review, Remote Sens. 14 (2022) , no. 4, 1–40.
[37] V. Sharma and R.N. Mir, A comprehensive and systematic look up into deep learning based object detection
techniques: a review, Comput. Sci. Rev. 38 (2020), 100301.
[38] K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 3rd Int. Conf.
Learn. Represent. ICLR 2015 - Conf. Track Proc. 2015, pp. 1–14.
[39] Z. Soleimanitaleb, M.A. Keyvanrad and A. Jafari, Object tracking methods: a review, 9th Int. Conf. Comput.
Knowl. Eng. ICCKE 2019, pp. 282–288.
[40] P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine and
V. Vasudevan, Scalability in perception for autonomous driving: waymo open dataset, Proc. IEEE Comput. Soc.
Conf. Comput. Vis. Pattern Recognit. 2020, pp. 2443–2451.
[41] A. U¸car, Y. Demir and C. G¨uzeli¸s, Object recognition and detection with deep learning for autonomous driving
applications, Simulation 93 (2017), no. 9, 759–769.
[42] K.E.A. Van De Sande, J.R.R. Uijlings, T. Gevers and A.W.M. Smeulders, Segmentation as selective search for
object recognition, Proc. IEEE Int. Conf. Comput. Vis. 2011, no. 2, pp. 1879–1886.
[43] M. Waheed, M. Javeed and A. Jalal, A novel deep learning model for understanding two-person interactions using
depth sensors, Int. Conf. Innov. Comput. (ICIC), IEEE, 2022, 1–8.
[44] T. Wollmann and K. Rohr, Deep consensus network: aggregating predictions to improve object detection in microscopy images, Med. Image Anal. 70 (2021), 102019.
[45] Y. Yin, Design of deep learning based autonomous driving control algorithm, 2nd Int. Conf. Consumer Electron.
Comput. Engin. (ICCECE), IEEE. 2022, pp. 423–426.
[46] Z. Zhang, Y. Li, W. Wu, H. Chen, L. Cheng and S. Wang, Tumor detection using deep learning method in
automated breast ultrasound, Biomed. Signal Process. Control 68 (2021), 102677.
[47] H.Y. Zhou, C. Wang, H. Li, G. Wang, S. Zhang, W. Li and Y. Yu, SSMD: semi-supervised medical image detection
with adaptive consistency and heterogeneous perturbation, Med. Image Anal. 72 (2021), 102117.
)
Volume 13, Issue 2
July 2022Pages 1495-1504
- Receive Date: 14 March 2022
- Revise Date: 19 April 2022
- Accept Date: 10 March 2022