Large scale objects thermography and thermal imaging survey: Datasets and applications

Document Type : Research Paper

Authors

Computer Science Department, College of Science, University of Diyala, Diyala, Baqubah, Iraq

Abstract

Due to machine learning-based infrared image and video collections, which enable computers to detect and categorize images with increasing accuracy, the field of image identification has experienced a revolution. This comprehensive research presents an overview of the most current advancements in the infrared image and video collections for computer vision and artificial intelligence. It has largely focused on the infrared picture and video collections that have been collected and categorized for computer vision applications such as object identification, object segmentation and classification, and motion detection. This article covers some of the most well-known machine learning methods, including deep learning, convolutional neural networks, support-vector machines, and decision trees. The basic problems with image identification are examined, and only a few of them include data augmentation, feature extraction, and picture segmentation. We also discuss some recent developments in the area of image identification, including ground-breaking deep learning methods like adversarial training and transfer learning. The discussion ends with a discussion of possible uses and the promise of machine learning for picture identification. Because it analyzes state of the art in machine learning for picture identification in-depth, this survey study is a vital resource for academics and entrepreneurs. We make a distinction between publicly accessible collections and those that are maintained in private, based on the various sensor types, image resolution, size, and research effort within that range. Include a glossary of words, including those for infrared radiation, infrared detectors, and infrared optics, that are crucial to comprehending infrared imaging, along with a description of their applications. This article explores the group's overall statistical relevance from a number of different perspectives. Researchers working in computer vision and artificial intelligence who are interested in managing spectra outside of the optical field might use this survey as a reference.

Keywords

[1] M. Abdrakhmanova, A. Kuzdeuov, S. Jarju, Y. Khassanov, M. Lewis and H.A. Varol, Speaking Faces: a large-scale multimodal dataset of voice commands with visual and thermal video streams, Sensors 21 (2021), no. 10, p. 3465.
[2] T. Afouras, J.S. Chung, A. Senior, O. Vinyals and A. Zisserman, Deep audio-visual speech recognition, IEEE Trans. Pattern Anal. Machine Intel. 44 (2018), no. 12, 8717–8727.
[3] H. Al-Sadr, M. Popescu and J.M. Keller, Early sepsis recognition based on infrared thermography, Int. J. Data Min. Bioinf. 22 (2019), no. 4, 301–327.
[4] R. Ashrafi, M. Azarbayjani and H. Tabkhi, A novel fully annotated thermal infrared face dataset: recorded in various environment conditions and distances from the camera, Infrared Phys. Technol. 124 (2022).
[5] A. Barnawi, P. Chhikara, R. Tekchandani, N. Kumar and B. Alzahrani, Artificial intelligence-enabled internet of things-based system for COVID-19 screening using aerial thermal imaging, Future Generat. Comput. Syst. 124 (2021), 119–132.
[6] M.L. Brioschi, C.D. Neto, M. de Toledo, E.B. Neves, J.V.C. Vargas and M.J. Teixeira, Infrared image method for possible COVID-19 detection through febrile and subfebrile people screening, J. Thermal Bio. 112 (2023), 103444.
[7] Q. Cai, D. Gallup, C. Zhang and Z. Zhang, 3d deformable face tracking with a commodity depth camera, Comput. Vision–ECCV 2010: 11th Eur. Conf. Comput. Vision, Heraklion, Crete, Greece, Proc. Part III 11, Springer Berlin Heidelberg, 2010, pp. 229–242.
[8] Z. Chen, S. Wang and Y. Qian, Multi-modality matters: A performance leap on VoxCeleb, INTERSPEECH, October 25–29, 2020, Shanghai, China, 2020, pp. 2252–2256.
[9] C.E. Cooper and P.C. Withers, Postural, pilo-erective and evaporative thermal windows of the short-beaked echidna (Tachyglossus aculeatus), Bio. Lett. 19 (2023), no. 1, p. 20220495.
[10] V. Espinosa-Duro, M. Faundez-Zanuy and J. Mekyska, A new face database simultaneously acquired in visible, near-infrared and thermal spectrums, Cognit. Comput. 5 (2013), 119–135.
[11] R. Gade and T.B. Moeslund, Thermal cameras and applications: A survey, Mach. Vis. Appl. 25 (2014), 245–262.
[12] S. Garrido-Jurado, R. Mu˜noz-Salinas, F.J. Madrid-Cuevas and M.J. Marın-Jimenez, Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognit. 47 (2014), 2280–2292.
[13] R.S. Ghiass, H. Bendada and X. Maldague, Universite laval face motion and time-lapse video database (ULFMTV), Technical Report; Universite Laval: Quebec, QC, Canada, 2018.
[14] R.I. Hammoud, IEEE OTCBVS WS series bench, http://vcipl-okstate.org/pbvs/bench/.
[15] G.B. Huang, M. Mattar, T. Berg and E. Learned-Miller, Labeled faces in the wild: A database for studying face recognition in unconstrained environments, In Workshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008.
[16] P. Katte, S.T. Kakileti, H.J. Madhu and G. Manjunath, Automated thermal screening for COVID-19 using machine learning, Artific. Intell. over Infrared Images Med. Appl. Med. Image Assisted Biomarker Discovery, MIABID AIIIMA 2022, Lecture Notes in Computer Science, Springer, Cham, 2022.
[17] Z. Ma, H. Li, W. Fang, Q. Liu, B. Zhou and Z. Bu, A cloud-edge-terminal collaborative system for temperature measurement in COVID-19 prevention, IEEE INFOCOM 2021-IEEE Conf. Comput. Commun. Workshops (INFOCOM WKSHPS), IEEE, 2021, pp. 1–6.
[18] A.C. Makino Antunes, A. Aldred, G.P. Tirado Moreno, J.A. de Souza Ribeiro, P.E. Brand˜ao, G.T. Barone, J.D.A. Conselheiro, A.C. Goulart, I.C. Desuo and G. Gomes, Potential of using facial thermal imaging in patient triage of flu-like syndrome during the COVID-19 pandemic crisis, PloS one 18 (2023), no. 1, 0279930.
[19] K. Mallat and J.L. Dugelay, A benchmark database of visible and thermal paired face images across multiple variations, Int. Conf. Biomet. Spec. Interest Group (BIOSIG), IEEE, 2018, pp. 1–5.
[20] K. Panetta, Q. Wan, S. Agaian, S. Rajeev, S. Kamath, R. Rajendran, S.P. Rao, A. Kaszowska, H.A. Taylor, A. Samani and X. Yuan, A comprehensive database for benchmarking imaging systems, IEEE Trans. Pattern Anal. Mach. Intell. 42 (2018), 509–520.
[21] D. Poster, M. Thielke, R. Nguyen, S. Rajaraman, X. Di, C.N. Fondje, V.M. Patel, N.J. Short, B.S. Riggan, N.M. Nasrabadi and S. Hu, A large-scale, time-synchronized visible and thermal face dataset, Proc. IEEE/CVF Winter Conf. Appl. Comput. Vision, 2021, pp. 1559–1568.
[22] S. Ren, K. He, R. Girshick and J. Sun, Faster r-cnn: Towards real-time object detection with region proposal networks, Adv. Neural Info. Process. Syst. 28 (2015), 91–99.
[23] D.J. R-FCN, Object detection via region-based fully convolutional networks, Adv. Neural Inf. Process. Syst., 2016, pp. 379–387.
[24] A. Scarano, F. Inchingolo and F. Lorusso, Facial skin temperature and discomfort when wearing protective face masks: thermal infrared imaging evaluation and hands moving the mask, Int. J. Environ. Res. Public Health 17 (2020), no. 13, 4624.
[25] S. Shon, T. Oh and J. Glass, Noise-tolerant audio-visual online person verification using an attention-based neural network fusion, ICASSP 2019-2019 IEEE Int. Conf. Acoustics, Speech and Signal Process. (ICASSP), IEEE, 2019, pp. 3995–3999.
[26] R.S.S. Singh, T.J.S. Anand, S.A. Anas and B. Acharya, A real-time analytic face thermal recognition system integrated with email notification, Eng. Technol. Appl. Sci. Res. 13 (2023), no. 1, 9961–9967.
[27] S. Wang, Z. Liu, S. Lv, Y. Lv, G. Wu, P. Peng, F. Chen and X. Wang, A natural visible and infrared facial expression database for expression recognition and emotion inference, IEEE Trans. Multimed. 12 (2010), 682–691.
[28] World Health Organization (WHO), https://www.who.int.
[29] W. Zhou, Y. Zhu, J. Lei, R. Yang and L. Yu, LSNet: Lightweight spatial boosting network for detecting salient objects in RGB-thermal images, EEE Trans. Image Process. 32 (2023), 1329–1340.
Volume 14, Issue 6
June 2023
Pages 109-118
  • Receive Date: 02 February 2023
  • Revise Date: 16 March 2023
  • Accept Date: 05 April 2023