ACLM Model: A CNN-LSTM and Machine Learning Approach for Analyzing Tourist Satisfaction to Improve Priority Tourism Services

Ulya Ilhami Arsyah, Mutiana Pratiwi, Harfeby Fryonanda, M. Khairul Anam, Munawir Munawir

Abstract


Tourist satisfaction is a key proxy for destination service quality, yet automatic sentiment analysis of online reviews still faces class imbalance, overfitting, and limited deployability. This study proposes ACLM, a hybrid sentiment classification pipeline that learns semantic and temporal features with a CNN-LSTM backbone and evaluates three classifier heads (Softmax, Logistic Regression, XGBoost) on a three-class corpus (neutral, satisfied, dissatisfied). The objective is to deliver an accurate and operational model for decision support in tourism services. The idea combines Word2Vec embeddings, a compact CNN for local patterns, an LSTM for sequence dependencies, and a training workflow with text cleaning, SMOTE based balancing, and regularization to curb overfitting; outputs are exposed through a simple Streamlit interface. Results show that CNN-LSTM with a Softmax head attains accuracy 0.89, macro precision 0.89, macro recall 0.84, and macro F1 0.86, outperforming Logistic Regression (accuracy 0.87, macro precision 0.84, macro recall 0.82, macro F1 0.82) and XGBoost (accuracy 0.85, macro precision 0.80, macro recall 0.82, macro F1 0.80). The findings indicate that deep sequence features paired with a simple Softmax head provide the best tradeoff between accuracy and stability for three-way sentiment classification. The contribution is a reusable, end to end blueprint from preprocessing and balanced training to quantitative evaluation and an inference GUI, and the novelty lies in testing interchangeable classifier heads on a single CNN-LSTM feature extractor while explicitly addressing data imbalance and deployment constraints. The GUI is implemented using the highest accuracy model, namely CNN-LSTM with Softmax.


Keywords


CNN-LSTM; Machine Learning; Streamlit; Tourist Satisfaction.

Full Text:

PDF

References


S. Ma and L. Zhang, “Enhancing Tourists’ Satisfaction: Leveraging Artificial Intelligence in the Tourism Sector,” Pacific International Journal, vol. 7, no. 3, pp. 89–98, Jun. 2024, doi: 10.55014/pij.v7i3.624.

B. A. Utari et al., “Tourist destination choice on five priority destinations of Indonesia during health crisis,” Journal of Destination Marketing and Management, vol. 32, pp. 1–12, Jun. 2024, doi: 10.1016/j.jdmm.2024.100880.

S. Z. Pranida and A. Kurniawardhani, “Sentiment Analysis of Expedition Customer Satisfaction using BiGRU and BiLSTM,” Indonesian Journal of Artificial Intelligence and Data Mining, vol. 5, no. 1, p. 44, Jun. 2022, doi: 10.24014/ijaidm.v5i1.17361.

M. K. Anam, S. Defit, Haviluddin, L. Efrizoni, and M. B. Firdaus, “Early Stopping on CNN-LSTM Development to Improve Classification Performance,” Journal of Applied Data Sciences, vol. 5, no. 3, pp. 1175–1188, 2024, doi: 10.47738/jads.v5i3.312.

Susandri, A. Zamsuri, N. Nasution, Y. Efendi, and H. B. Alwan, “Mitigating Overfitting in Sentiment Analysis Insights from CNN-LSTM Hybrid Models,” MATRIK: Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer, vol. 24, no. 2, pp. 297–308, Mar. 2025, doi: 10.30812/matrik.v24i2.4742.

N. El Koufi, Y. M. Missah, and A. Belangour, “A Hybrid CNN-LSTM Based Natural Language Processing Model for Sentiment Analysis of Customer Product Reviews: A Case Study from Ghana,” Journal of Hunan University Natural Sciences, vol. 51, no. 8, pp. 59–71, 2024, doi: 10.55463/issn.1674-2974.51.8.5.

A. Nasser, H. Zaied, and G. Soliman, “Arabic Sentiment Analysis using Deep Learning and Machine Learning approaches,” Journal of Computing and Communication, vol. 3, no. 2, pp. 10–22, 2024, doi: 10.21608/jocc.2024.380113.

M. F. Madjid, D. E. Ratnawati, and B. Rahayudi, “Sentiment Analysis on App Reviews Using Support Vector Machine and Naïve Bayes Classification,” Sinkron, vol. 8, no. 1, pp. 556–562, Feb. 2023, doi: 10.33395/sinkron.v8i1.12161.

M. Sakhdiah, A. Salma, D. Permana, and D. Fitria, “Sentiment Analysis Using Support Vector Machine (SVM) of ChatGPT Application Users in Play Store,” UNP Journal of Statistics and Data Science, vol. 2, no. 2, pp. 151–158, May 2024, doi: 10.24036/ujsds/vol2-iss2/158.

A. E. Minarno, M. Fadhlan, Y. Munarko, and R. Chandranegara, “Classification of Dermoscopic Images Using CNN-SVM,” International of Dermoscopic Images Using CNN-SVM, vol. 3, no. 2, pp. 606–612, 2024, doi: 10.62527/joiv.8.2.2153.

D. Deepthiswarupa, G. Vignesh, K. Lasya Chowdary, R. D. Ingale, and A. professor, “A Hybrid Deep Learning and Machine Learning Approach for Sentiment-Enhanced Movie Recommendations,” International Journal for Research Trends and Innovation, vol. 10, no. 5, p. 530, 2025, [Online]. Available: www.ijrti.org

M. K. Anam, S. Sumijan, K. Karfindo, and M. B. Firdaus, “Comparison Analysis of HSV Method, CNN Algorithm, and SVM Algorithm in Detecting the Ripeness of Mangosteen Fruit Images,” Indonesian Journal of Artificial Intelligence and Data Mining, vol. 7, no. 2, pp. 348–356, May 2024, doi: 10.24014/ijaidm.v7i2.29739.

M. K. Anam et al., “Sara Detection on Social Media Using Deep Learning Algorithm Development,” Journal of Applied Engineering and Technological Science, vol. 6, no. 1, pp. 225–237, Dec. 2024, doi: 10.37385/jaets.v6i1.5390.

H. Handoko, A. Asrofiq, J. Junadhi, and A. S. Negara, “Sentiment Analysis of Sirekap Tweets Using CNN Algorithm,” INTENSIF: Jurnal Ilmiah Penelitian dan Penerapan Teknologi Sistem Informasi, vol. 8, no. 2, pp. 312–329, Aug. 2024, doi: 10.29407/intensif.v8i2.23046.

N. Leelawat et al., “Twitter data sentiment analysis of tourism in Thailand during the COVID-19 pandemic using machine learning,” Heliyon, vol. 8, no. 10, pp. 1–11, Oct. 2022, doi: 10.1016/j.heliyon.2022.e10894.

A. D. Poernomo and S. Suharjito, “Indonesian online travel agent sentiment analysis using machine learning methods,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 14, no. 1, pp. 113–117, Apr. 2019, doi: 10.11591/ijeecs.v14.i1.pp113-117.

C. Xu, P. Coen-Pirani, and X. Jiang, “Empirical Study of Overfitting in Deep Learning for Predicting Breast Cancer Metastasis,” Cancers (Basel), vol. 15, no. 7, pp. 1–18, Apr. 2023, doi: 10.3390/cancers15071969.

Hartono and E. Ongko, “Avoiding Overfitting dan Overlapping in Handling Class Imbalanced Using Hybrid Approach with Smoothed Bootstrap Resampling and Feature Selection,” International Journal on Informatics Visualization, vol. 2, no. 6, pp. 343–348, 2022, doi: 10.30630/joiv.6.2.985.

Y. A. Singgalen, “A Hybrid CNN-LSTM Model with SMOTE for Enhanced Sentiment Analysis of Hotel Reviews,” Technology and Science (BITS), vol. 6, no. 3, pp. 1363–1373, 2024, doi: 10.47065/bits.v6i3.6301.

A. Alasmari, N. Farooqi, and Y. Alotaibi, “Sentiment analysis of pilgrims using CNN-LSTM deep learning approach,” PeerJ Comput Sci, vol. 10, pp. 1–47, 2024, doi: 10.7717/PEERJ-CS.2584.

X. T. Dang and T. T. Le, “KNN-SMOTE: An Innovative Resampling Technique Enhancing the Efficacy of Imbalanced Biomedical Classification,” in Machine Learning and Other Soft Computing Techniques: Biomedical and Related Applications, N. Hoang Phuong, N. T. Huyen Chau, and V. Kreinovich, Eds., Cham: Springer Nature Switzerland, 2024, pp. 111–121. doi: 10.1007/978-3-031-63929-6_11.

N. Matondang and N. Surantha, “Effects of oversampling SMOTE in the classification of hypertensive dataset,” Advances in Science, Technology and Engineering Systems, vol. 5, no. 4, pp. 432–437, 2020, doi: 10.25046/AJ050451.

M. A. Latief, L. R. Nabila, W. Miftakhurrahman, S. Ma’rufatullah, and H. Tantyoko, “Handling Imbalance Data using Hybrid Sampling SMOTE-ENN in Lung Cancer Classification,” International Journal of Engineering and Computer Science Applications (IJECSA), vol. 3, no. 1, pp. 11–18, Feb. 2024, doi: 10.30812/ijecsa.v3i1.3758.

D. Datta et al., “A Hybrid Classification of Imbalanced Hyperspectral Images Using ADASYN and Enhanced Deep Subsampled Multi-Grained Cascaded Forest,” Remote Sens (Basel), vol. 14, no. 19, Oct. 2022, doi: 10.3390/rs14194853.

R. J. Dhanal and V. R. Ghorpade, “Aspect-based sentiment-analysis using topic modelling and machine-learning,” International Journal of Electrical and Computer Engineering, vol. 14, no. 6, pp. 6689–6698, Dec. 2024, doi: 10.11591/ijece.v14i6.pp6689-6698.

S. Agarwal, “Deep Learning-based Sentiment Analysis: Establishing Customer Dimension as the Lifeblood of Business Management,” Global Business Review, vol. 23, no. 1, pp. 119–136, Feb. 2022, doi: 10.1177/0972150919845160.

P. H. Prastyo, I. Ardiyanto, and R. Hidayat, “Indonesian Sentiment Analysis: An Experimental Study of Four Kernel Functions on SVM Algorithm with TF-IDF,” in International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), Institute of Electrical and Electronics Engineers Inc., Oct. 2020, pp. 1–6. doi: 10.1109/ICDABI51230.2020.9325685.

H. Basri, M. B. S. Junianto, and I. Kusyadi, “Enhancing Usability Testing Through Sentiment Analysis: A Comparative Study Using SVM, Naive Bayes, Decision Trees and Random Forest,” Jurnal Teknologi Sistem Informasi dan Aplikasi, vol. 7, no. 4, pp. 1603–1610, Oct. 2024, doi: 10.32493/jtsi.v7i4.45117.

A. C. M. V. Srinivas, Ch. Satyanarayana, Ch. Divakar, and K. P. Sirisha, “Sentiment Analysis using Neural Network and LSTM,” in IOP Conference Series: Materials Science and Engineering, IOP Publishing, Feb. 2021, pp. 1–7. doi: 10.1088/1757-899x/1074/1/012007.

Z. Abidin, A. Junaidi, and Wamiliana, “Text Stemming and Lemmatization of Regional Languages in Indonesia: A Systematic Literature Review,” Journal of Information Systems Engineering and Business Intelligence, vol. 10, no. 2, pp. 217–231, Jun. 2024, doi: 10.20473/jisebi.10.2.217-231.

S. Priyadarshinee and M. Panda, “CARDIAC DISEASE PREDICTION USING SMOTE AND MACHINE LEARNING CLASSIFIERS,” Journal of Pharmaceutical Negative Results ¦, vol. 13, no. 8, p. 2022, 2022, doi: 10.47750/pnr.2022.13.S08.108.

G. Husain et al., “SMOTE vs. SMOTEENN: A Study on the Performance of Resampling Algorithms for Addressing Class Imbalance in Regression Models,” Algorithms, vol. 18, no. 1, pp. 1–16, Jan. 2025, doi: 10.3390/a18010037.

A. Fellah, A. Zahaf, and A. Elçi, “Semantic Similarity Measure Using a Combination of Word2Vec and WordNet Models,” Indonesian Journal of Electrical Engineering and Informatics, vol. 12, no. 2, pp. 455–464, Jun. 2024, doi: 10.52549/ijeei.v12i2.5114.

V. Yadav, P. Verma, and V. Katiyar, “Enhancing sentiment analysis in Hindi for E-commerce companies: a CNN-LSTM approach with CBoW and TF-IDF word embedding models,” International Journal of Information Technology (Singapore), pp. 1–16, 2023, doi: 10.1007/s41870-023-01596-x.

P. K. Jain, V. Saravanan, and R. Pamula, “A Hybrid CNN-LSTM: A Deep Learning Approach for Consumer Sentiment Analysis Using Qualitative User-Generated Contents,” ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 20, no. 5, pp. 1–15, Sep. 2021, doi: 10.1145/3457206.

Q. Zhu, Z. He, T. Zhang, and W. Cui, “Improving classification performance of softmax loss function based on scalable batch-normalization,” Applied Sciences (Switzerland), vol. 10, no. 8, Apr. 2020, doi: 10.3390/APP10082950.

A. Desfiandi and B. Soewito, “Student Graduation Time Prediction Using Logistic Regression, Decision Tree, Support Vector, And Adaboost Ensemble Learning,” International Journal of Information System and Computer Science) IJISCS, vol. 7, no. 3, pp. 195–199, 2023, doi: 10.56327/ijiscs.v7i2.1579.

M. A. Permana, M. I. Thohir, T. Mantoro, and M. A. Ayu, “Crime Rate Detection Based on Text Mining on Social Media Using Logistic Regression Algorithm,” in 7th International Conference on Computing, Engineering and Design, ICCED 2021, Institute of Electrical and Electronics Engineers Inc., 2021. doi: 10.1109/ICCED53389.2021.9664846.

D. Tarwidi, S. R. Pudjaprasetya, D. Adytia, and M. Apri, “An optimized XGBoost-based machine learning method for predicting wave run-up on a sloping beach,” MethodsX, vol. 10, no. 1, pp. 1–12, Aug. 2023, doi: 10.1145/2939672.2939785.

T. Sugihartono, B. Wijaya, Marini, A. F. Alkayes, and H. A. Anugrah, “Optimizing Stunting Detection through SMOTE and Machine Learning: a Comparative Study of XGBoost, Random Forest, SVM, and k-NN,” Journal of Applied Data Sciences, vol. 6, no. 1, pp. 667–682, Jan. 2025, doi: 10.47738/jads.v6i1.494.

C. W. Bartlett et al., “Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data,” SN Comput Sci, vol. 4, no. 2, pp. 1–12, Mar. 2023, doi: 10.1007/s42979-022-01553-8.

M. Vilares Ferro, Y. Doval Mosquera, F. J. Ribadas Pena, and V. M. Darriba Bilbao, “Early stopping by correlating online indicators in neural networks,” Neural Networks, vol. 159, pp. 109–124, Feb. 2023, doi: 10.1016/j.neunet.2022.11.035.

K. L. Tan, C. P. Lee, and K. M. Lim, “A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research,” Applied Sciences (Switzerland), vol. 13, no. 7, pp. 1–21, Apr. 2023, doi: 10.3390/app13074550.

A. Saidi, S. Ben Othman, and S. Ben Saoud, “Hybrid CNN-SVM classifier for efficient depression detection system,” in Proceedings of the International Conference on Advanced Systems and Emergent Technologies, IC_ASET 2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020, pp. 229–234. doi: 10.1109/IC_ASET49463.2020.9318302.

F. Liman, C. Carsten, S. Sufinata, S. Irviantina, and S. Winardi, “Implementation of BiLSTM-SVM Algorithm to Detect Fake News on Text-Based Media,” Jurnal CoreIT, vol. 9, no. 2, pp. 27–34, Jan. 2024, doi: 10.24014/coreit.v9i2.18982.

I. Evelyn Ezhilarasi and J. C. Clement, “GRU-SVM Based Threat Detection in Cognitive Radio Network,” Sensors, vol. 23, no. 3, pp. 1–17, Feb. 2023, doi: 10.3390/s23031326.

L. Khan, A. Amjad, K. M. Afaq, and H. T. Chang, “Deep Sentiment Analysis Using CNN-LSTM Architecture of English and Roman Urdu Text Shared in Social Media,” Applied Sciences (Switzerland), vol. 12, no. 5, Mar. 2022, doi: 10.3390/app12052694.




DOI: https://doi.org/10.47738/jads.v6i4.974

Refbacks

  • There are currently no refbacks.



Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0