.png)
Sentimental Analysis of Legal Aid Services: A Machine Learning Approach
Abstract
Legal Aid services in South Africa, administered by Legal Aid South Africa (SA), aim to provide essential legal representation to vulnerable individuals lacking financial resources. Despite its significant role, there is a pervasive perception among the public that the quality of these state-funded services is substandard, often leading to negative attitudes towards the organization. This research employs sentiment analysis to evaluate client perceptions of Legal Aid SA's services, using a dataset of 5,246 entries from Twitter and the Internal client feedback system between 2019 and 2024. The study utilizes various machine learning algorithms, including Naive Bayes, Stochastic Gradient Descent (SGD), Random Forest, Support Vector Classification (SVC), Logistic Regression, and Extreme Gradient Boosting (XGBoost), to analyze sentiment polarity and classify feedback into positive, neutral, and negative sentiments. The accuracy, precision, recall, and F1 scores assessed model performance. The SVC and XGBoost models demonstrated superior performance, achieving testing accuracies of 90.10% and 90.00%, respectively. In contrast, Naive Bayes and Logistic Regression lagged, with test accuracies of 82.00% and 85.00%, respectively. The findings reveal that most responses are either neutral or positive, suggesting a predominantly favourable impression of Legal Aid services. This research not only aims to enhance Legal Aid SA's service offerings but may also provide valuable insights for similar organizations globally.
Keywords
Full Text:
PDFReferences
J. Makokoane, D. Khosa, and B. Obrenovic, “Applying UTAUT and Fuzzy Dematel Methods: A New Legal Aid Administration System,” THE INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE AND BUSINESS ADMINISTRATION, vol. 8, no. 1, pp. 24–36, Nov. 2021, doi: 10.18775/ijmsba.1849-5664-5419.2014.81.1002.
Shunmuga Lakshmi Priya. K, Thamarai Selvi. D, S. Kalaiselvi, and V. Gomathi, “A Sentimental Analysis of Legal Documents using Deep Learning Approach,” 2022 International Conference on Automation, Computing and Renewable Systems (ICACRS), Dec. 2022, doi: 10.1109/icacrs55517.2022.10029322.
Puviyarasi Thirugnanasammandamoorthi, Harsh Kumar, Debabrata Ghosh, Chandramohan Dhasarathan, and Ram Kishan Dewangan, “Sentimental analysis and prediction of socioeconomic disasters tweets by ML and regular expression,” Journal of Intelligent & Fuzzy Systems, 2024, doi: 10.3233/jifs-219417.
Bola Abimbola, Enrique A. De La Cal Marín, and Qing Tan, “Enhancing Legal Sentiment Analysis: A Convolutional Neural Network-Long Short-Term Memory Document-Level Model,” Machine Learning and Knowledge Extraction, 2024, doi: 10.3390/make6020041.
I Wayan Budi Suryawan, Nengah Widya Utami, and Ketut Queena Fredlina, “ANALISIS SENTIMEN REVIEW WISATAWAN PADA OBJEK WISATA UBUD MENGGUNAKAN ALGORITMA SUPPORT VECTOR MACHINE,” Jurnal Informatika, Teknologi dan Sains, vol. 5, no. 1, pp. 133–140, Feb. 2023, doi: 10.51401/jinteks.v5i1.2242.
“Einblick- A Sentimental Analysis And Opinion Mining System For Mobile Networks,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 10, no. 3, pp. 2165–2174, Jun. 2021, doi: 10.30534/ijatcse/2021/931032021.
S Sumayah, Falentino Sembiring, and Wisuda Jatmiko, “ANALYSIS OF SENTIMENT OF INDONESIAN COMMUNITY ON METAVERSE USING SUPPORT VECTOR MACHINE ALGORITHM,” Jurnal Teknik Informatika (Jutif), vol. 4, no. 1, pp. 143–150, Feb. 2023, doi: 10.52436/1.jutif.2023.4.1.417.
None Mohammed Athar Rangila et al., “Sentimental Analysis using Bert Algorithm over LSTM,” International Journal of Advanced Research in Science, Communication and Technology, pp. 455–459, Oct. 2022, doi: 10.48175/ijarsct-7300.
R. Pramana, Debora, J. Subroto, A. Gunawan, and A. Anderies, Systematic Literature Review of Stemming and Lemmatization Performance for Sentence Similarity. 2022, p. 6. doi: 10.1109/ICITDA55840.2022.9971451.
J. Khosa, D. Mashao, A. Olanipekun, and C. Harley, “How Effective are Different Machine Learning Algorithms in Predicting Legal Outcomes in South Africa?,” Journal of Applied Data Sciences, vol. 5, no. 4, Art. no. 4, Oct. 2024, doi: 10.47738/jads.v5i4.215.
D. Khyani, B. S. Siddhartha, N. M. Niveditha, and B. M. Divya, “An interpretation of lemmatization and stemming in natural language processing,” Journal of University of Shanghai for Science and Technology, vol. 22, no. 10, pp. 350–357, 2021.
R. Egger and E. Gokce, “Natural Language Processing (NLP): An Introduction: Making Sense of Textual Data,” 2022, pp. 307–334. doi: 10.1007/978-3-030-88389-8_15.
J. Jefriyanto, N. Ainun, and M. A. A. Ardha, “Application of Naïve Bayes Classification to Analyze Performance Using Stopwords,” Journal of Information System, Technology and Engineering, vol. 1, no. 2, pp. 49–53, Jun. 2023, doi: 10.61487/jiste.v1i2.15.
A. I. Kabir, K. Ahmed, and R. Karim, “Word Cloud and Sentiment Analysis of Amazon Earphones Reviews with R Programming Language,” Informatica Economica, vol. 24, pp. 55–71, Dec. 2020, doi: 10.24818/issn14531305/24.4.2020.05.
R. K. Bania, “COVID-19 Public Tweets Sentiment Analysis using TF-IDF and Inductive Learning Models,” INFOCOMP Journal of Computer Science, vol. 19, no. 2, Art. no. 2, Dec. 2020.
S. N. Khan, S. U. Khan, H. Aznaoui, C. B. Şahin, and Ö. B. Dinler, “Generalization of linear and non-linear support vector machine in multiple fields: a review,” Computer Science and Information Technologies, vol. 4, no. 3, Art. no. 3, Nov. 2023, doi: 10.11591/csit.v4i3.pp226-239.
“Research and application of XGBoost in imbalanced data.” Accessed: Oct. 15, 2024. [Online]. Available: https://journals.sagepub.com/doi/epub/10.1177/15501329221106935
S. Wu, Q. Yuan, Z. Yan, and Q. Xu, “Analyzing Accident Injury Severity via an Extreme Gradient Boosting (XGBoost) Model,” Journal of Advanced Transportation, vol. 2021, no. 1, p. 3771640, 2021, doi: 10.1155/2021/3771640.
B. Charbuty, Adnan Abdulazeez, Adnan Abdulazeez, and A. M. Abdulazeez, “Classification Based on Decision Tree Algorithm for Machine Learning,” Journal of Applied Science and Technology Trends, vol. 2, no. 1, pp. 20–28, 2021, doi: 10.38094/jastt20165.
O. ElSahly and A. Abdelfatah, “An Incident Detection Model Using Random Forest Classifier,” Smart Cities, vol. 6, no. 4, Art. no. 4, Aug. 2023, doi: 10.3390/smartcities6040083.
M. Mahdikhani, “Predicting the popularity of tweets by analyzing public opinion and emotions in different stages of Covid-19 pandemic,” International Journal of Information Management Data Insights, vol. 2, no. 1, p. 100053, 2022, doi: 10.1016/j.jjimei.2021.100053.
A. Sharma, “Guided Stochastic Gradient Descent Algorithm for inconsistent datasets,” Applied Soft Computing Journal, vol. 73, pp. 1068–1080, 2018, doi: 10.1016/j.asoc.2018.09.038.
Y. Wang, D. Qiu, Y. Wang, M. Sun, and G. Strbac, “Graph Learning-Based Voltage Regulation in Distribution Networks With Multi-Microgrids,” IEEE Transactions on Power Systems, vol. 39, no. 1, pp. 1881–1895, Jan. 2024, doi: 10.1109/TPWRS.2023.3242715.
U. Gazder, A. Ahmed, and U. Shahid, “Predicting Severity of Accidents in Malaysia By Ordinal Logistic Regression Models,” JTTM, vol. 03, no. 01, pp. 11–16, Mar. 2021, doi: 10.5383/JTTM.03.01.002.
S. Agrawal, S. Jain, S. Sharma, A. K.-I. J. of, and undefined 2023, “COVID-19 Public Opinion: A Twitter Healthcare Data Processing Using Machine Learning Methodologies,” mdpi.com.
D. E. Cahyani and I. Patasik, “Performance comparison of TF-IDF and Word2Vec models for emotion text classification,” Bulletin of Electrical Engineering and Informatics, vol. 10, no. 5, Art. no. 5, Oct. 2021, doi: 10.11591/eei.v10i5.3157.
Hubert, P. Phoenix, R. Sudaryono, and D. Suhartono, “Classifying Promotion Images Using Optical Character Recognition and Naïve Bayes Classifier,” Procedia Computer Science, vol. 179, pp. 498–506, Jan. 2021, doi: 10.1016/j.procs.2021.01.033.
V. Balakrishnan and W. Kaur, “String-based Multinomial Naïve Bayes for Emotion Detection among Facebook Diabetes Community,” Procedia Computer Science, vol. 159, pp. 30–37, Jan. 2019, doi: 10.1016/J.PROCS.2019.09.157.
A. Kibriya, E. Frank, B. Pfahringer, G. H.-A. 2004: A. in, and undefined 2005, “Multinomial naive bayes for text categorization revisited,” Springer.
I. Priyadarshini, P. Mohanty, R. Kumar, R. Sharma, V. Puri, and P. K. Singh, “A study on the sentiments and psychology of twitter users during COVID-19 lockdown period,” Multimedia Tools and Applications, vol. 81, no. 19, pp. 27009–27031, 2022, doi: 10.1007/s11042-021-11004-w.
DOI: https://doi.org/10.47738/jads.v6i2.521
Refbacks
- There are currently no refbacks.
Journal of Applied Data Sciences
| ISSN | : | 2723-6471 (Online) |
| Publisher | : | Bright Publisher |
| Website | : | http://bright-journal.org/JADS |
| : | taqwa@amikompurwokerto.ac.id (principal contact) | |
| support@bright-journal.org (technical issues) |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0