International Journal of Nonlinear Analysis and Applications

Integration of deep learning model and feature selection for multi-label classification

Document Type : Research Paper

Authors

1 Department of IT and Computer Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran

2 Assistant Professor, Department of IT and Computer Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran

Abstract

Multi-label data classification differs from traditional single-label data classification, in which each input sample participated with just one class tag. As a result of the presence of multiple class tags, the learning process is affected, and single-label classification can no longer be used. Methods for changing this problem have been developed. By using these methods, one can run the usual classifier classes on the data. Multi-label classification algorithms are used in a variety of fields, including text classification and semantic image annotation. A novel multi-label classification method based on deep learning and feature selection is presented in this paper with specific meta-label-specific features. The results of experiments on different multi-label datasets demonstrate that the proposed method is more efficient than previous methods.

Keywords

[1] Z. Al-Makhadmeh and A. Tolba, Utilizing IoT wearable medical device for heart disease prediction using higher order Boltzmann model: A classification approach, Measurement 147 (2019) 106815.
[2] M.R. Boutell, J. Luo, X. Shen and C.M. Brown, Learning multi-label scene classification, Pattern Recogn. 37(9) (2004) 1757–1771.
[3] H. Chao, X. Fang, J. Zhang, F. Homayounieh, C.D. Arru, S.R. Digumarthy, R. Babaei, H.K. Mobin, I. Mohseni, L. Saba, A. Carriero, Z. Falaschi, A. Pasche, G. Wang, M.K. Kalra and P. Yan, Integrative analysis for COVID-19 patient outcome prediction, Med. Image Anal. 67 (2021) 101844.
[4] H. Chen, W. Cai, F. Wu and Q. Liu, Vehicle-mounted far-infrared pedestrian detection using multi-object tracking, Infrared Phys. Tech. 115 (2021) 103697.
[5] J. Djolonga, J. Yung, M. Tschannen, R. Romijnders, L. Beyer, A. Kolesnikov, J. Puigcerver, M. Minderer, A. D’Amour, D. Moldovan and S. Gelly, On robustness and transferability of convolutional neural networks, Proc. IEEE/CVF Conf. Computer Vision and Pattern Recogn. (2021) 16458–16468.
[6] H. Dong, J. Sun, X. Sun and R. Ding, A many-objective feature selection for multi-label classification, KnowledgeBased Syst. 208 (2020) 106456.
[7] W.A. Ezat, M.M. Dessouky and N.A. Ismail, Evaluation of deep learning YOLOv3 algorithm for object detection and classification, Menoufia J. Elect. Eng. Res. 30(1) (2021) 52–57.
[8] A. Hashemi, M.B. Dowlatshahi and H. Nezamabadi-Pour, MGFS: A multi-label graph-based feature selection algorithm via PageRank centrality, Expert Syst. Appl. 142 (2020) 113024.
[9] J. Hu, Y. Li, W. Gao and P. Zhang, Robust multi-label feature selection with dual-graph regularization, KnowledgeBased Syst. 203 (2020) 106126.
[10] B. Jeong, N. Ko, C. Son and J. Yoon, Trademark-based framework to uncover business diversification opportunities: Application of deep link prediction and competitive intelligence analysis, Comput. Indust. 124 (2021) 103356.
[11] Z. Ji, B. Cui, H. Li, Y.-G. Jiang, T. Xiang, T. Hospedales and Y. Fu, Deep ranking for image zero-shot multi-label classification, IEEE Trans. Image Proces. 29 (2020) 6549–6560.
[12] L. Kalake, W. Wan and L. Hou, Analysis based on recent deep learning approaches applied in real-time multi-object tracking: A review, IEEE Access 9 (2021) 32650–32671.
[13] C. Khammassi and S. Krichen, A GA-LR wrapper approach for feature selection in network intrusion detection, Comput. Secur. 70 (2017) 255–277.
[14] S. Khandagale, H. Xiao and R. Babbar, Bonsai: diverse and shallow trees for extreme multi-label classification, Machine Learn. 109(11) (2020) 2099–2119.
[15] S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj and D.J. Inman, 1D convolutional neural networks and applications: A survey, Mech. Syst. Signal Sroces. 151 (2021) 107398.
[16] Z. Li, F. Liu, W. Yang, S. Peng and J. Zhou, A survey of convolutional neural networks: analysis, applications, and prospects, IEEE Trans. Neural Networks Learn. Syst. (2021) 1–21.
[17] M. Mafarja and S. Mirjalili, Whale optimization approaches for wrapper feature selection, Appl. Soft Comput. 62 (2018) 441–453.
[18] F. Morais-Rodrigues, R. Silverio, R.K. Kato, D.L.N. Rodrigues, J. Valdez-Baez, V. Fonseca, E.J. San, L.G.R. Gomes, R.G. dos Santos, M.V.C. Viana, J.d.C. Ferraz Dutra, M.T.D. Parise, D. Parise, F.F. Campos, S.J. de Souza, J.M. Ortega, D. Barh, P. Ghosh and M.A. dos Santos, Analysis of the microarray gene expression for breast cancer progression after the application modified logistic regression, Gene 726 (2020) 144168.
[19] A.A. Ojugo and O. Nwankwo, Spectral-cluster solution for credit-card fraud detection using a genetic algorithm trained modular deep learning neural network, J. Inf. Visual. 2(1) (2021) 15–24.
[20] D. Paul, A. Jain, S. Saha and J. Mathew, Multi-objective PSO based online feature selection for multi-label classification, Knowledge-Based Syst. 222 (2021) 106966.
[21] P. Prajapati and A. Thakkar, Performance improvement of extreme multi-label classification using K-way tree construction with parallel clustering algorithm, J. King Saud Univ. Comput. Info. Sci., (2021).
[22] I. Rejer and M. Twardochleb, Gamers’ involvement detection from EEG data with cGAAM – A method for feature selection for clustering, Expert Syst. Appl. 101 (2018) 196–204.
[23] M.S. Salman, Y. Du, D. Lin, Z. Fu, A. Fedorov, E. Damaraju, J. Sui, J. Chen, A.R. Mayer, S. Posse, D.H. Mathalon, J.M. Ford, T.V. Erp and V.D. Calhoun, Group ICA for identifying biomarkers in schizophrenia: ‘Adaptive’ networks via spatially constrained ICA show more sensitivity to group differences than spatiotemporal regression, NeuroImage: Clinic. 22 (2019) 101747.
[24] D. Song, A. Vold, K. Madan and F. Schilder, Multi-label legal document classification: A deep learning-based approach with label-attention and domain-specific pre-training, Inf. Syst. (2021) 101718.
[25] Q.Z. Song, L. Zhao, X.K. Luo and X.C. Dou, Using deep learning for classification of lung nodules on computed tomography images, J. Healthcare Engin. 2017 (2017) Article ID 8314740.
[26] F.A. Thabtah, P. Cowling, Y. Peng, R. Rastogi, K. Morik, M. Bramer and X. Wu, MMAC: A new multi-class, multi-label associative classification approach, Proc. Fourth IEEE Int. Conf. Data Min. 2004 (2004) 217–224.
[27] W. Wang, Q.Y. Dai, F. Li, Y. Xiong and D.-Q. Wei, MLCDForest: multi-label classification with deep forest in disease prediction for long non-coding RNAs, Brief. Bioinf. 22(3) (2020).
[28] R. Wang, S. Kwong, X. Wang and Y. Jia, Active k-label sets ensemble for multi-label classification, Pattern Recogn. 109 (2021) 107583.
[29] D. Wang, M. Li, N. Ben-Shlomo, C.E. Corrales, Y. Cheng, T. Zhang and J. Jayender, A novel dual-network architecture for mixed-supervised medical image segmentation, Comput. Med. Imag. Graph. 89 (2021) 101841.
[30] T. Wu, Q. Huang, Z. Liu, Y. Wang and D. Lin, Distribution-balanced loss for multi-label classification in longtailed datasets, A. Vedaldi, H. Bischof, T. Brox and J.M. Frahm, Computer Vision – ECCV 2020. ECCV 2020, Lecture Notes in Computer Science, Springer, 12349 (2020).
[31] Y. Xia, K. Chen and Y. Yang, Multi-label classification with weighted classifier selection and stacked ensemble, Info. Sci. 557 (2021) 421–442.
[32] J. Xu, An extended one-versus-rest support vector machine for multi-label classification, Neurocomput. 74(17) (2011) 3114–3124.
[33] M. Yang, W. Zhao, L. Chen, Q. Qu, Z. Zhao and Y. Shen, Investigating the transferring capability of capsule networks for text classification, Neural Networks 118 (2019) 247–261.
[34] X.H. Yap and M. Raymer, Multi-label classification and label dependence in silico toxicity prediction, Toxicol. Vitro 74 (2021) 105157.
[35] R. You, Z. Guo, L. Cui, X. Long, Y. Bao and S. Wen, Cross-modality attention with semantic graph embedding for multi-label classification, Proc. AAAI Conf. Artificial Intell. (2020).
[36] D. Yun, J. Ryu and J. Lim, Dual aggregated feature pyramid network for multi-label classification, Pattern Recogn. Lett. 144 (2021) 75–81.
[37] R. Zeleznik, B. Foldyna, P. Eslami, J. Weiss, I. Alexander, J. Taron, C. Parmar, R.M. Alvi, D. Banerji, M. Uno and Y. Kikuchi, Deep convolutional neural networks to predict cardiovascular risk from computed tomography, Nature Commun. 12(1) (2021) 1–9.
[38] Y. Zhang, Y. Wang, X.-Y. Liu, S. Mi and M.-L. Zhang, Large-scale multi-label classification using unknown streaming images, Pattern Recogn.99 (2020) 107100.
[39] W. Zhao, F. Chen, H. Huang, D. Li and W. Cheng, A new steel defect detection algorithm based on deep learning, Comput. Intell. Neurosci. (2021).
[40] X. Zheng, W. Zhu, C. Tang and M. Wang, Gene selection for microarray data classification via adaptive hypergraph embedded dictionary learning, Gene 706 (2019) 188–200.
[41] Y. Zhong, B. Du and C. Xu, Learning to reweight examples in multi-label classification, Neural Networks 142 (2021) 428–436.
International Journal of Nonlinear Analysis and Applications
Volume 13, Issue 1
March 2022
Pages 2871-2883
  • Receive Date: 10 October 2021
  • Revise Date: 28 November 2021
  • Accept Date: 26 December 2021