Hybrid CNN--ELM model for robust post-cosmetic surgery face recognition

Document Type : Research Paper

Authors
1 Department of Electrical Engineering, Isf.C., Islamic Azad University, Isfahan, Iran
2 Department of Electrical Engineering, Mo.C., Islamic Azad University, Mobarakeh, Isfahan, Iran
Abstract
Cosmetic surgery introduces substantial structural changes to facial features, often leading to reduced accuracy in conventional face recognition systems. To address this challenge, we present a hybrid recognition framework that integrates Convolutional Neural Networks (CNNs) for deep feature extraction with an Extreme Learning Machine (ELM) for classification. The CNN component captures surgery-invariant hierarchical features, while the ELM provides faster convergence and improved generalization compared to fully connected layers. The proposed method was evaluated using the IIIT-Delhi Plastic Surgery Face Database, which includes 1,800 images from 900 subjects across 13 types of local and global procedures. Results indicate significant performance gains across all key metrics. For local surgeries, the model achieved average Recall, Precision, and Accuracy of 95.3%, 95.1%, and 95.2%, respectively, compared to 81.3%, 81.2%, and 82.6% for a conventional CNN. Specificity and F-measure improved to 94.6% and 94.8%, reflecting enhanced reliability and reduced false positives. For global procedures such as rhytidectomy and skin resurfacing, the model maintained accuracy above 93%, exceeding the baseline by over 10%. These findings demonstrate that the CNN--ELM hybrid model offers superior robustness to post-surgical variations, making it a promising solution for biometric verification in forensic investigations, healthcare applications, and security systems.
Keywords

[1] R.R. Atallah, A.S. Al-Shamayleh, and M.A. Awadallah, Face plastic surgery recognition model based on neural network and meta-learning model, J. Univ. Comput. Sci. 29 (2023), no. 10.
[2] A. Bharati, R. Singh, M. Vatsa, and K.W. Bowyer, Detecting facial retouching using supervised deep learning, IEEE Trans. Inf. Forensics Secur. 11 (2016), no. 9, 1903–1913.
[3] H.S. Bhatt, S. Bharadwaj, R. Singh, and M. Vatsa, Recognizing surgically altered faces using hybrid SURF and neural network approaches, Proc. IEEE Conf. Comput. Vision Pattern Recogn. Workshops, 2012, 47–54.
[4] E. Borsting, R. DeSimone, M. Ascha, and M. Ascha, Applied deep learning in plastic surgery: classifying rhinoplasty with a mobile app, J. Craniofac. Surgery 31 (2020), no. 1, 102–106.
[5] J. Bouguila and H. Khochtali, Facial plastic surgery and face recognition algorithms: Interaction and challenges, A scoping review and future directions, J. Plastic Reconstruct. Aesthetic Surgery 74 (2020), no. 1, 217–228.
[6] H. Chen, Z. Qin, Y. Ding, L. Tian, and Z. Qin, Brain tumor segmentation with deep convolutional symmetric neural network, Neurocomputing 392 (2020), 305–313.
[7] P. Dadure, S. Sikder, and K. Sambyo, Plastic surgery face recognition using LBP and PCA algorithm, 2nd Int. Conf. Electron. Mater. Engin. Nano-Technol., 2018, 1–5.
[8] N. Damer, J.H. Grebe, F. Kirchbuchner, and A. Uhl, Impact of plastic surgery on face recognition: Analysis with state-of-the-art algorithms and high-quality datasets, Proc. IEEE Int. Joint Conf. Biomet., 2021, pp. 1–8.
[9] A. Devi and A. Marimuthu, An efficient self-updating face recognition system for plastic surgery face, ICTACT J. Image Video Process. 7 (2016), no. 1, 1325–1331.
[10] M. Ebadi, H. Rashidy Kanan, and M. Kalantari, Face recognition using patch manifold learning across plastic surgery from a single training exemplar per enrolled person, Signal Image Video Process. 14 (2020), 1071–1077.
[11] S.A. El-Said and H.M. Abol Atta, Geometrical face recognition after plastic surgery, Int. J. Comput. Appl. Technol. 49 (2014), no. 3, 4, 352–364.
[12] R.M. Ibrahim, F.E.Z. Abou-Chadi, and A.S. Samra, Plastic surgery face recognition: A comparative study of performance, Int. J. Comput. Sci. Issues 8 (2011), no. 6, 1–7.
[13] R. Khedgaonkar, K. Singh, and M. Raghuwanshi, Local plastic surgery-based face recognition using convolutional neural networks, Demyst. Big Data, Machine Learn. Deep Learn. Healthcare Anal., 2020, pp. 1–15.
[14] X. Liu, S. Shan, and X. Chen, Face recognition after plastic surgery: A comprehensive study, Asian Conf. Comput. Vision, 2012, pp. 565–576.
[15] P. Mlynarski, H. Delingette, A. Criminisi, and N. Ayache, 3D convolutional neural networks for tumor segmentation using long-range 2D context, Comput. Med. Imag. Graph. 73 (2019), 60–72.
[16] R. Nogueira, M. Eguchi, J. Kasmirski, B.V. de Lima, D.C. Dimatos, D.L. Lima, R. Glatter, D.L. Tran, and P.S. Piccinini, Machine learning, deep learning, artificial intelligence and aesthetic plastic surgery: A qualitative systematic review, Aesthetic Plastic Surgery 49 (2025), no. 1, 389–399.
[17] M.A. Ranjanikar and U.V. Kulkarni, Novel face recognition framework for plastic surgery and unconstrained facial datasets, Int. J. Engin. Adv. Technol. 9 (2020), no. 4, 1234–1240.
[18] C. Rathgeb, D. Dogan, F. Stockhardt, M. De Marsico, and C. Busch, Plastic surgery: An obstacle for deep face recognition?, Proc. IEEE/CVF Conf. Comput. Vision Pattern Recogn. Workshops, 2020.
[19] C. Rathgeb, C.I. Satnoianu, N.E. Haryanto, K. Bernardo, N. Damer, C. Busch, and R. Veldhuis, Impact of facial beautification on face recognition: A comprehensive study, IEEE Access 10 (2022), 49630–49650.
[20] T. Sabharwal, Human identification after plastic surgery using region based score level fusion of local facial features, PhD Thesis, Thapar Institute of Engineering & Technology, 2019.
[21] T. Sabharwal and R. Gupta, Deep facial recognition after medical alterations, Multimedia Tools Appl. 81 (2022), no. 18, 25675–25706.
[22] A.H. Sable, S.N. Talbar, and H.A. Dhirbasi, Recognition of plastic surgery faces and the surgery types: An approach with entropy based scale invariant features, J. King Saud Univ.-Comput. Inf. Sci. 31 (2019), no. 4, 554–560.
[23] R. Singh, M. Vatsa, H.S. Bhatt, S. Bharadwaj, A. Noore, and S.S. Nooreyezdan, Plastic surgery: A new dimension to face recognition, IEEE Trans. Inf. Forensics Secur. 5 (2010), no. 3, 441–448.
[24] D. Sinha, J.P. Pandey, and B. Chauhan, Soft computing-based approach for face recognition on plastic surgery and skin colour-based scenarios using CNN, IOT Smart Syst.: Proc. ICTIS 2022 2 (2022), 363–374.
[25] R. Singh, M. Vatsa, and A. Noore, Effect of plastic surgery on face recognition: A critical survey and empirical study, Pattern Recogn. Lett. 31 (2010), no. 10, 1267–1276.
[26] B.S. Sruthy and M. Jayasree, Recognizing surgically altered face images and 3D facial expression recognition, Procedia Technol. 24 (2016), 1300–1304.
[27] M.M. Zhang, W.J. Di, T. Song, N.B. Yin, and Y.Q. Wang, Exploring artificial intelligence from a clinical perspective: A comparison and application analysis of two facial age predictors trained on a large-scale Chinese cosmetic patient database, Skin Res. Technol. 29 (2023), no. 7, e13402.
[28] Y. Zhang, X. Wang, and J. Li, Machine learning applications in facial cosmetic surgeries: A scoping review, J. Med. Artific. Intell. 6 (2023), 12–25.

Articles in Press, Corrected Proof
Available Online from 17 November 2026

  • Receive Date 22 September 2025
  • Revise Date 07 October 2025
  • Accept Date 24 October 2025