International Journal of Nonlinear Analysis and Applications
Ensemble deep learning for aspect-based sentiment analysis
Document Type : Research Paper
Authors
1 Assistant Professor, Computer Department, University of Isfahan, Isfahan, Iran.
2 Graduate student, IT and Computer Department, Sepahan Institute of Higher Education, Isfahan, Iran.
Abstract
Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.
Keywords
SemEval2016- Task 5: SVM and CRF for Aspect Detection and Unsupervised Aspect-Based Sentiment Analysis, in: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego,
California, (2016) pp. 306–311.
[2] S. Ardabili and A. Musavi, Advances in Machine Learning Modeling Reviewing Hybrid and Ensemble Methods,
Lecture Notes in Networks and Systems. 101 (2019) 215–227.
[3] R. Arulmurugan, K.R. Sabarmathi and H. Anandakumar, Classification of sentence level sentiment analysis using
cloud machine learning techniques, Cluster Computing. 22 (2019) 1199–1209.
[4] M. Awni, M.I. Khalil and H.M. Abbas, Deep-Learning Ensemble for Offline Arabic Handwritten Words Recognition, in: 2019 14th International Conference on Computer Engineering and Systems (ICCES, Cairo, Egypt,(
2019) pp. 40–45.
[5] E. Camberia, Affective computing and sentiment analysis, IEEE Intelligent Systems. 31 (2016) 102–107.
[6] E. Can, A. Ezen-can and F. Can, Multilingual Sentiment Analysis: An RNN-Based Framework for Limited Data,
in: ACM SIGIR 2018 Workshop on Learning from Limited or Noisy Data, (2018).
[7] Y. Cheng, L. Yao, G. Xiang, G. Zhang, T. Tang and L. Zhong, Text Sentiment Orientation Analysis Based on
Multi-Channel CNN and Bidirectional GRU With Attention Mechanism, IEEE Access. 8 (2020) 134964–134975.
[8] S. Clematide, A Simple and Effective biLSTM Approach to Aspect-Based Sentiment Analysis in Social Media
Customer Feedback, in: 14th Conference on Natural Language Processing, Vienna, Austria, 2018: pp. 29–33.
[9] H.H. Do, P. Prasad, A. Maag and A. Alsadoon, Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review, Expert Systems With Applications. 118 (2019) 272–299.[10] Y. Freund, RESchapire, A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,
Journal of Computer and System Sciences. 55 (1997) 119–139.
[11] D. Gamal, M. Alfonse, E.-S. M. El-Horbaty and A.-B. M. Salem, Analysis of Machine Learning Algorithms for
Opinion Mining in Different Domains, Machine Learning and Knowledge Extraction. 1 (2019) 224–234.
[12] P. Liu, S. Joty and H. Meng, Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon,
Portugal, (2015) pp. 1433–1443.
[13] G. Rao, W. Huang, Z. Feng and Q. Cong, LSTM with sentence representations for document-level sentiment
classification, Neurocomputing. 308 (2018) 49–57.
[14] O. Sagi and L. Rokach, Ensemble learning: A survey, Data Mining and Knowledge Discovery. 8 (2018) 1–18.
[15] K. Sarkar, A Stacked Ensemble Approach to Bengali Sentiment Analysis, Lecture Notes in Computer Science.
11886 (2020) 102–111.
[16] A. Sherstinsky, Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM)
Network, Physica D: Nonlinear Phenomena. 404 (2020) 1–43.
[17] K. Smagulova and A.P. James, A survey on LSTM memristive neural network architectures and applications,
The European Physical Journal Special Topics. 228 (2019) 2313–2324.
[18] Y. Wang, M. Huang, L. Zhao and X. Zhu, Attention-based LSTM for Aspect-level Sentiment Classification, in:
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, (2016)
pp. 606–615.
[19] Y. Xing and C. Xiao, A GRU Model for Aspect Level Sentiment Analysis, Journal of Physics: Conference Series.
1302 (2019) 1–7.
[20] G. Xu, Y. Meng, X. Qiu, Z. Yu and X. Wu, Sentiment Analysis of Comment Texts Based on BiLSTM, IEEE
Access. 7 (2019) 51522–51532.
[21] H. Xu, B. Liu, L. Shu and P.S. Yu, Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne,
Australia, (2018) pp. 592–598.
[22] A. Yadav and D.K. Vishwakarma, Sentiment analysis using deep learning architectures: a review, Artificial
Intelligence Review. 53 (2020) 4335–4385.
[23] F. Yang, C. Du and L. Huang, Ensemble Sentiment Analysis Method based on R-CNN and C-RNN with Fusion
Gate, International Journal of Computers Communications & Control. 14 (2019) 272–285.
[24] S. Yang, X. Yu and Y. Zhou, LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp
Review Dataset as an Example, in: 2020 International Workshop on Electronic Communication and Artificial
Intelligence (IWECAI), IEEE, Shanghai, China, (2020) pp. 98–101.
[25] L. Zhang, S. Wang and B. Liu, Deep learning for sentiment analysis: a survey, data mining and knowledge
dscovery. 8 (2018) 1–34.
[26] D.-X. Zhou, Universality of Deep Convolutional Neural, Applied and Computational Harmonic Analysis. 48
(2020) 787–794.
[27] X. Zhou, X. Wan and J. Xiao, Attention-based LSTM Network for Cross-Lingual Sentiment Classification, in:
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, (2016)
pp. 247–256.
[28] SemEval2016 task5, BhMad Studio. (2016). Available: http://alt.qci.org/semeval2016/task5
[29] SemEval2014 task4 (2014). Available: http://alt.qcri.org/semeval2014/task4/
)
Volume 12, Special Issue
December 2021Pages 29-38
- Receive Date: 03 October 2020
- Revise Date: 07 December 2020
- Accept Date: 29 December 2020