Object-Level Sentiment Analysis Use a Language Model

Thuy Thi Le, Tuoi Thi Phan

Abstract


Sentiment analysis remains a prominent area of research in the natural language processing (NLP) community and holds significant practical value in domains such as commerce and education. Most existing approaches evaluate sentiments for a single object or product, typically categorizing them as positive or negative. However, when a text involves comparisons between multiple objects, it can be challenging to identify which sentiment or emotion is associated with which object. Few studies have addressed this issue, often stopping at evaluating emotions at the sentence level or for individual words related to aspects or objects. This study proposes an object-level sentiment analysis problem that produces a set of pairs or triples consisting of an object, aspect, and sentiment. Additionally, in texts expressing opinions or comments on a specific aspect, the aspect may be implied through references to the object without being explicitly mentioned. Identifying such implicit aspects is crucial, as it ensures no loss of information and enhances the efficiency of extraction of information in object-level sentiment analysis. The integration of implicit aspect identification and object-level sentiment analysis is the primary focus of this research. In recent years, many language models have been developed and effectively applied to various NLP tasks. Therefore, to address the proposed challenges, this study utilizes deep learning that incorporates language models combined with NLP methods such as parsing and dependency analysis, to achieve the desired output. Using language model and NLP techniques automatically generate training data for the learning model. The proposed method achieves an accuracy of 90%, making a substantial contribution to the field of NLP.


Keywords


Language Model; Deep Learning; Implicit Aspect; Sentiment; Object-Level Sentiment Analysis

Full Text:

PDF

References


V.A. Kharde, S. Sonawane, “Sentiment analysis of Twitter data: a survey of techniques,” International Journal of Computer Applications, vol 139(11), pp. 5-15, 2016.

O. Appel, F. Chiclana, J Carter, H Fujita, “A hybrid approach to the sentiment analysis problem at the sentence level,” Knowledge-Based Systems, vol 108, pp. 110-124, 2016.

X Fang, J Zhan, “Sentiment analysis using product review data,” Journal of Big Data, vol 2, pp. 1-14, 2015, https://doi.org/10.1186/s40537-015-0015-2.

X. Fu, W. Liu, Y. Xu, C. Yu, T. Wang, “Long short-term memory network over rhetorical structure theory for sentence-level sentiment analysis,” in Proceeding of the Asian conference on machine learning, Nov 16-18, Hamilton, New Zealand, pp 17-32, 2016.

M.S. Akhtar, D. Gupta, A. Ekbal, P. Bhattacharyya, “Feature selection and ensemble construction: A two-step method for aspect based sentiment analysis,” Knowledge-Based Systems, vol 125, pp. 116-135, 2017.

W. Chuhan, W. Fangzhao, W. Sixing, Y. Zhigang, L. Junxin and H. Yongfeng, "Semi-supervised dimensional sentiment analysis with variational autoencoder," Knowledge-Based Systems, vol. 165, pp. 30-39, 2019.

S. Kumar, M. Gahalawat, P. P. Roy, D. P. Dogra and B.-G. Kim, "Exploring Impact of Age and Gender on Sentiment Analysis Using Machine Learning," Electronics, vol. 9:374, no. 2, 2020.

G. Alexandridis, J. Aliprantis, K. Michalakis, K. Korovesis, P. Tsantilas, and G. Caridakis, “A Knowledge-Based Deep Learning Architecture for Aspect-Based Sentiment Analysis,” Int J Neural Syst, vol. 31, no. 10, Aug. 2021.

Q. Le and T. Mikolov, "Distributed Representations of Sentences and Documents," in Proceedings of the 31st International Conference on Machine Learning, June 22-24, Bejing, China, pp. 1188-1196, 2014.

Z. Xinjie, W. Xiaojun and X. Jianguo, "Attention-based LSTM Network for Cross-Lingual Sentiment Classification," in EMNLP, 2016.

Y. Yin, Y. Song and M. Zhang, "Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension," in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 7-11, Copenhagen, Denmark, pp. 2044-2054, 2017.

D. Tang and M. Zhang, "Deep learning in sentiment analysis," in Deep Learning in Natural Language Processing, Berlin, Springer, 2018, pp. 219-253.

M. Schmitt, S. Steinheber, K. Schreiber and B. Roth, "Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, October 31 - Novemer 4, Brussels, Belgium, pp. 1109-1114, 2018.

H. D. Hai, P. Prasad, M. Angelika and A. Abeer, "Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review," Expert Systems with Applications, vol. 118, pp. 272-299, 2019.

S. Soni, A. Sharaff, “Sentiment analysis of customer reviews based on hidden Markov model,” in Proceedings of the 2015 international conference on advanced research in computer science engineering & technology (ICARCSET 2015), March 6, Unnao, India, pp. 1-5, 2015.

M.P Salas-Zàrate, J. Medina-Moreira, K. Lagos-Ortiz, H. Luna-Aveiga, M.A. Rodriguez-Garcia, R.J.C. ValenciaGarcía, “Sentiment analysis on tweets about diabetes: an aspect-level approach,” Computational and Mathematical Methods in Medicine, vol 2017, pp. 1-9, 2017.

M.S. Akhtar, A. Kumar, A. Ekbal, P. Bhattacharyya, “A hybrid deep learning architecture for sentiment analysis,” in Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Dec, Osaka, Japan, pp. 482-493, 2016.

A.C. Pandey, D.S. Rajpoot, M. Saraswat, “Twitter sentiment analysis using hybrid cuckoo search method,” Information Processing & Management, vol 53, no. 4, pp. 764-779, 2017.

X. Ding and B. Liu, "Resolving Object and Attribute Coreference in Opinion Mining," in Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), August 23-27, Beijing, China, pp. 268-276, 2010.

Y. Zhao, B. Qin, T. Liu and W. Yang, "Aspect-Object Alignment with Integer Linear Programming in Opinion Mining," PLOS ONE, vol. 10, no. 5, pp. 1-18, 2015.

P. K. Soni and R. Rambola, “A Survey on Implicit Aspect Detection for Sentiment Analysis: Terminology, Issues, and Scope,” IEEE Access, vol. 10, pp. 63932–63957, Jun. 2022.

O. M. AL-Janabi, N. H. Ahamed Hassain Malim, and Y. N. Cheah, “Unsupervised model for aspect categorization and implicit aspect extraction,” Knowledge and Information Systems 2022 64:6, vol. 64, no. 6, pp. 1625–1651, Apr. 2022.

B. Liu, "Sentiment analysis and opinion mining," Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1--167, 2012

M. Hu and B. Liu, "Mining and summarizing customer reviews," in Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, Seattle, Washington, USA, pp. 168-177, 2004.

T. A. Rana and Y.-N. Cheah, "Aspect Extraction in Sentiment Analysis: Comparative Analysis and Survey," Artif. Intell. Rev., vol. 46, no. 4, p. 459–483, 2016.

V. Ganganwar and R. Rajalakshmi, "Implicit Aspect Extraction for Sentiment Analysis: A Survey of Recent Approaches," Procedia Comput. Sci., vol. 165, no. C, p. 485–491, 2019.

Q. Su, X. Xu, H. Guo, Z. Guo, X. Wu, X. Zhang, B. Swen and Z. Su, "Hidden sentiment association in chinese web opinion mining," in Proceedings of the 17th international conference on World Wide Web, April 21-25, Beijing, China, pp. 959-968, 2008.

L. Zeng and F. Li, "A classification-based approach for implicit feature identification," in Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, LNAI 8202, October 10-12, Suzhou, China, pp. 190-202, 2013.

L. Chen, J. Martineau, D. Cheng and A. Sheth, "Clustering for simultaneous extraction of aspects and features from reviews," in proceedings of the 2016 conference of the north American chapter of the Association for Computational Linguistics: human language technologies, June 12-17, San Diego, California, pp. 789–799, 2016.

J. Yu, Z.-J. Zha, M. Wang, K. Wang and T.-S. Chua, "Domain-assisted product aspect hierarchy generation: towards hierarchical organization of unstructured consumer reviews," in Proceedings of the 2011 conference on empirical methods in natural language processing, July 27-31, Edinburgh, Scotland, UK, pp. 140-150, 2011.

E. H. Hajar and B. Mohammed, "Hybrid approach to extract adjectives for implicit aspect identification in opinion mining," in 2016 11th International Conference on Intelligent Systems: Theories and Applications (SITA), October 19-20, Mohammedia, Morocco, pp. 1-5, 2016.

J. Feng, S. Cai and X. Ma, "Enhanced sentiment labeling and implicit aspect identification by integration of deep convolution neural network and sequential algorithm," Cluster Computing, vol. 22, no. 3, pp. 5839-5857, 2019.

A. Bagheri, M. Saraee and F. De Jong, "Care more about customers: Unsupervised domain-independent aspect detection for sentiment analysis of customer reviews," Knowledge-Based Systems, vol. 52, pp. 201-213, 2013.

T. A. Rana, Y.-N. Cheah and T. Rana, "Multi-level knowledge-based approach for implicit aspect identification," Applied Intelligence, vol. 50, no. 12, pp. 4616-4630, 2020.

T. L. Thi, T. P. Thi, & T. Q. Thanh, Machine learning using context vectors for object coreference resolution. Computing 105, 539–558 (2023). https://doi.org/10.1007/s00607-021-00902-4.

T. L. Thi, T. K. Tran and T. T. Phan, "Deep Learning Using Context Vectors to Identify Implicit Aspects," in IEEE Access, vol. 11, pp. 39385-39393, 2023, doi: 10.1109/ACCESS.2023.3268243.

Devlin, M.-W. Chang, K. Lee and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2-7, Minneapolis, MN, USA, pp. 4171-4186, 2019.

T. L. Thi, T. Q. Thanh, and T. P. Thi, “Ontology-based entity coreference resolution for sentiment analysis”, in SoICT 2017: Proceedings of the Eighth International Symposium on Information and Communication Technology, Dec., vol. 2017-Decem, pp. 50–56, 2017.




DOI: https://doi.org/10.47738/jads.v6i1.448

Refbacks

  • There are currently no refbacks.



Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0