Hybrid Transformer-XGBOOST Model Optimized with Ant Colony Algorithm for Early Heart Disease Detection: A Risk Factor-Driven and Interpretable Method

Moch Deny Pratama, Faris Abdi El Hakim, Dimas Novian Aditia Syahputra, Dodik Arwin Dermawan, Asmunin Asmunin, Salamun Rohman Nudin, Andi Iwan Nurhidayat
Pages 148-164
Abstract Views 766
PDF Downloads 544
Crossref Citations -
DOI https://doi.org/10.47738/jads.v7i1.969

Abstract


Cardiovascular diseases (CVDs) remain the leading cause of death worldwide, with significant socioeconomic consequences due to premature death and chronic disability. Although clinical screening techniques have evolved, early and accurate prediction of heart disease is still partial due to the limited capacity of conventional machine learning algorithms to model the complex nonlinear interactions among various contributing risk factors e.g., hypertension, diabetes, hyperlipidemia, and genetic predisposition. To address these challenges, this research introduces a hybrid framework that combines the Transformer architecture known for its robust self-attention mechanism and high representational capabilities with Ant Colony Optimization (ACO), a nature-inspired metaheuristic algorithm modeled on the foraging behavior of ants, to enable adaptive and efficient hyperparameter optimization. The proposed model processes structured clinical data by encoding categorical variables into embeddings and normalizing numerical features, resulting in a unified tabular representation suitable for transformer-based analysis. ACO improves model efficiency by optimizing key parameters e.g., embedding configuration, learning rate, and depth, reducing manual intervention and computational overhead. The proposed Hybrid Transformer-ACO model focuses on interpretable clinical features to provide actionable risk stratification. Model evaluation was performed using classification metrics e.g., accuracy, precision, recall, F1 score, and time complexity to measure predictive performance and computational efficiency during the training and inference phases. These evaluation criteria provide evidence of the model's diagnostic reliability, generalizability, and practical feasibility for clinical application.. The model achieved 100% accuracy, sensitivity, specificity, and F1-score, outperforming several models. Time complexity analysis demonstrated efficient training and testing, while the model interpretability supports transparency and trust.

Keywords


Cardiovascular Disease; Heart Disease Detection; Transformer; Ant Colony Optimization; Machine Learning

Full Text:

PDF

References


A. Naeem, S. H. Abbas, M. Yousaf, A. Ishtiaq, and I. Murtaza, “Global Impact and Strategies to Reduce the Mortality from Cardiovascular Diseases,” in Integrated Science for Sustainable Development Goal 3: Empowering Global Wellness Initiatives, Springer, 2024, pp. 283–306.

J. Kundu and S. Kundu, “Cardiovascular disease (CVD) and its associated risk factors among older adults in India: evidence from LASI Wave 1,” Clin. Epidemiol. Glob. Heal., vol. 13, p. 100937, 2022.

C. Fastl, A. Arnberger, V. Gallistl, V. K. Stein, and T. E. Dorner, “Heat vulnerability: health impacts of heat on older people in urban and rural areas in Europe,” Wien. Klin. Wochenschr., vol. 136, no. 17, pp. 507–514, 2024.

M. Vaduganathan, G. A. Mensah, J. V. Turco, V. Fuster, and G. A. Roth, “The global burden of cardiovascular diseases and risk: a compass for future health,” Journal of the American College of Cardiology, vol. 80, no. 25. American College of Cardiology Foundation Washington DC, pp. 2361–2371, 2022.

N. Merlina, A. Prasetio, I. Zuniarti, N. A. Mayangky, D. N. Sulistyowati, and F. Aziz, “Improving Early Detection of Cervical Cancer Through Deep Learning-Based Pap Smear Image Classification,” J. Appl. Data Sci., vol. 6, no. 2, pp. 952–968, 2025.

H. Y. Hassen, R. Ndejjo, J.-P. Van Geertruyden, G. Musinguzi, S. Abrams, and H. Bastiaens, “Type and effectiveness of community-based interventions in improving knowledge related to cardiovascular diseases and risk factors: a systematic review,” Am. J. Prev. Cardiol., vol. 10, p. 100341, 2022.

K. K. Teo and T. Rafiq, “Cardiovascular risk factors and prevention: a perspective from developing countries,” Can. J. Cardiol., vol. 37, no. 5, pp. 733–743, 2021.

M. Khalifa, M. Albadawy, and U. Iqbal, “Advancing clinical decision support: the role of artificial intelligence across six domains,” Comput. Methods Programs Biomed. Updat., p. 100142, 2024.

T. Kyriazos and M. Poga, “Application of machine learning models in social sciences: managing nonlinear relationships,” Encyclopedia, vol. 4, no. 4, pp. 1790–1805, 2024.

S. Nerella et al., “Transformers and large language models in healthcare: A review,” Artif. Intell. Med., p. 102900, 2024.

S. R. Choi and M. Lee, “Transformer architecture and attention mechanisms in genome data analysis: a comprehensive review,” Biology (Basel)., vol. 12, no. 7, p. 1033, 2023.

T. Lai, “Interpretable medical imagery diagnosis with self-attentive transformers: a review of explainable AI for health care,” BioMedInformatics, vol. 4, no. 1, pp. 113–126, 2024.

F. Yazdi and S. Asadi, “Enhancing Cardiovascular Disease Diagnosis: The Power of Optimized Ensemble Learning,” IEEE Access, vol. 13, pp. 46747–46762, 2025.

X. Li et al., “Artificial general intelligence for medical imaging analysis,” IEEE Rev. Biomed. Eng., 2024.

L. Žigutytė, T. Sorz-Nechay, J. Clusmann, and J. N. Kather, “Use of artificial intelligence for liver diseases: A survey from the EASL congress 2024,” JHEP Reports, p. 101209, 2024.

K. T. Chitty-Venkata, S. Mittal, M. Emani, V. Vishwanath, and A. K. Somani, “A survey of techniques for optimizing transformer inference,” J. Syst. Archit., vol. 144, p. 102990, 2023.

M. A. Awadallah et al., “Multi-objective ant colony optimization,” Arch. Comput. Methods Eng., pp. 1–43, 2024.

T. R. D. Saputri, E. Kurniawan, C. C. Lestari, and T. Antonio, “Nature-based Hyperparameter Tuning of a Multilayer Perceptron Algorithm in Task Classification: A Case Study on Fear of Failure in Entrepreneurship,” J. Appl. Data Sci., vol. 6, no. 2, pp. 969–980, 2025.

A. H. Fristiana, S. A. I. Alfarozi, A. E. Permanasari, M. Pratama, and S. Wibirama, “A Survey on Hyperparameters Optimization of Deep Learning for Time Series Classification,” IEEE Access, 2024.

G. S. Kumar and P. Kumaresan, “Deep Learning and Transfer Learning in Cardiology: A Review of Cardiovascular Disease Prediction Models,” IEEE Access, 2024.

L. S. Dhingra, M. Shen, A. Mangla, and R. Khera, “Cardiovascular care innovation through data-driven discoveries in the electronic health record,” Am. J. Cardiol., vol. 203, pp. 136–148, 2023.

D.-K. Kim and K. Kim, “A convolutional transformer model for multivariate time series prediction,” IEEE Access, vol. 10, pp. 101319–101329, 2022.

F. M. Mustafa et al., “TabNet and TabTransformer: Novel Deep Learning Models for Chemical Toxicity Prediction in Comparison With Machine Learning,” J. Appl. Toxicol., 2025.

I. Shaer, S. Nikan, and A. Shami, “Efficient Transformer-Based Hyper-Parameter Optimization for Resource-Constrained IoT Environments,” IEEE Internet Things Mag., 2024.

X. Zhou, H. Ma, J. Gu, H. Chen, and W. Deng, “Parameter adaptation-based ant colony optimization with dynamic hybrid mechanism,” Eng. Appl. Artif. Intell., vol. 114, p. 105139, 2022.

P. P. Putra, M. K. Anam, A. S. Chan, A. Hadi, N. Hendri, and A. Masnur, “Optimizing Sentiment Analysis on Imbalanced Hotel Review Data Using SMOTE and Ensemble Machine Learning Techniques,” J. Appl. Data Sci., vol. 6, no. 2, pp. 921–935, 2025.

N. A. Baghdadi, S. M. Farghaly Abdelaliem, A. Malki, I. Gad, A. Ewis, and E. Atlam, “Advanced machine learning techniques for cardiovascular disease early detection and diagnosis,” J. Big Data, vol. 10, no. 1, p. 144, 2023.

S. Hossain, M. K. Hasan, M. O. Faruk, N. Aktar, R. Hossain, and K. Hossain, “Machine learning approach for predicting cardiovascular disease in Bangladesh: evidence from a cross-sectional study in 2023,” BMC Cardiovasc. Disord., vol. 24, no. 1, p. 214, 2024.

P. Saha, A. De, S. D. Roy, and M. K. Bhowmik, “A comprehensive review on deep cardiovascular disease detection approaches: its datasets, image modalities and methods,” Multimed. Tools Appl., pp. 1–47, 2024.

A. Rahali and M. A. Akhloufi, “End-to-end transformer-based models in textual-based NLP,” Ai, vol. 4, no. 1, pp. 54–110, 2023.

Z. Wang and J. Sun, “Transtab: Learning transferable tabular transformers across tables,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 2902–2915, 2022.

M. M. Ahsan and Z. Siddique, “Machine learning-based heart disease diagnosis: A systematic literature review,” Artif. Intell. Med., vol. 128, p. 102289, 2022, doi: https://doi.org/10.1016/j.artmed.2022.102289.

P. Ghosh et al., “Efficient prediction of cardiovascular disease using machine learning algorithms with relief and LASSO feature selection techniques,” IEEE Access, vol. 9, pp. 19304–19326, 2021.

V. Shorewala, “Early detection of coronary heart disease using ensemble techniques,” Informatics Med. Unlocked, vol. 26, p. 100655, 2021.

A. Masitha, M. K. Biddinika, and H. Herman, “K Value Effect on Accuracy Using the K-NN for Heart Failure Dataset,” MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 22, no. 3, pp. 593–604, 2023.

V. Chang, V. R. Bhavani, A. Q. Xu, and M. A. Hossain, “An artificial intelligence model for heart disease detection using machine learning algorithms,” Healthc. Anal., vol. 2, p. 100016, 2022.

T. Jumphoo, K. Phapatanaburi, W. Pathonsuwan, P. Anchuen, M. Uthansakul, and P. Uthansakul, “Exploiting data-efficient image transformer-based transfer learning for valvular heart diseases detection,” IEEE Access, vol. 12, pp. 15845–15855, 2024.

A. B. Naeem et al., “Heart Disease Detection Using Feature Extraction and Artificial Neural Networks: A Sensor-Based Approach,” IEEE Access, vol. 12, pp. 37349–37362, 2024, doi: 10.1109/ACCESS.2024.3373646.

D. Zhou, H. Qiu, L. Wang, and M. Shen, “Risk prediction of heart failure in patients with ischemic heart disease using network analytics and stacking ensemble learning,” BMC Med. Inform. Decis. Mak., vol. 23, no. 1, p. 99, 2023.

S. Arooj, S. ur Rehman, A. Imran, A. Almuhaimeed, A. K. Alzahrani, and A. Alzahrani, “A deep convolutional neural network for the early detection of heart disease,” Biomedicines, vol. 10, no. 11, p. 2796, 2022.

M. Kavitha, G. Gnaneswar, R. Dinesh, Y. R. Sai, and R. S. Suraj, “Heart Disease Prediction using Hybrid machine Learning Model,” in 2021 6th International Conference on Inventive Computation Technologies (ICICT), 2021, pp. 1329–1333. doi: 10.1109/ICICT50816.2021.9358597.

E. M. Senan, I. Abunadi, M. E. Jadhav, and S. M. Fati, “Score and Correlation Coefficient‐Based Feature Selection for Predicting Heart Failure Diagnosis by Using Machine Learning Algorithms,” Comput. Math. Methods Med., vol. 2021, no. 1, p. 8500314, 2021.

M. Muntasir Nishat et al., “A Comprehensive Investigation of the Performances of Different Machine Learning Classifiers With SMOTE‐ENN Oversampling Technique and Hyperparameter Optimization for Imbalanced Heart Failure Dataset,” Sci. Program., vol. 2022, no. 1, p. 3649406, 2022.

A. Abdellatif, H. Abdellatef, J. Kanesan, C.-O. Chow, J. H. Chuah, and H. M. Gheni, “An effective heart disease detection and severity level classification model using machine learning and hyperparameter optimization methods,” ieee access, vol. 10, pp. 79974–79985, 2022.

A. G. Gad, “Particle swarm optimization algorithm and its applications: a systematic review,” Arch. Comput. methods Eng., vol. 29, no. 5, pp. 2531–2561, 2022.

L. Wu, X. Huang, J. Cui, C. Liu, and W. Xiao, “Modified adaptive ant colony optimization algorithm and its application for solving path planning of mobile robot,” Expert Syst. Appl., vol. 215, p. 119410, 2023.

A. K. Dubey, A. Kumar, and R. Agrawal, “An efficient ACO-PSO-based framework for data classification and preprocessing in big data,” Evol. Intell., vol. 14, pp. 909–922, 2021.

K. Brown et al., “Using Artificial Intelligence for Rheumatic Heart Disease Detection by Echocardiography: Focus on Mitral Regurgitation,” J. Am. Heart Assoc., vol. 13, no. 2, p. e031257, Jan. 2024, doi: 10.1161/JAHA.123.031257.

J. Zhao, L. Gao, and S. Ren, “Prediction of open-pit mine truck travel time based on LSTM-TabTransformer,” Sci. Rep., vol. 15, no. 1, p. 7427, 2025.

Y. Han, Z. Wei, and G. Huang, “An imbalance data quality monitoring based on SMOTE-XGBOOST supported by edge computing,” Sci. Rep., vol. 14, no. 1, p. 10151, 2024.

A. M. Ebid, M. Y. Abdel-Kader, I. M. Mahdi, and I. Abdel-Rasheed, “Ant Colony Optimization based algorithm to determine the optimum route for overhead power transmission lines,” Ain Shams Eng. J., vol. 15, no. 1, p. 102344, 2024.

M. D. Pratama, R. Abdillah, and D. Z. Haq, “Water Quality Identification Using Ensemble Machine Learning and Hybrid Resampling SMOTE-ENN Algorithm,” Fountain Informatics J., vol. 9, no. 2, pp. 83–91, 2024.

K. O. Lye, S. Mishra, D. Ray, and P. Chandrashekar, “Iterative surrogate model optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks,” Comput. Methods Appl. Mech. Eng., vol. 374, p. 113575, 2021.

L. Kyriakidis, M. Bähr, and M. A. Mendez, “Enhanced hybrid algorithm based on Bayesian optimization and Interior Point OPTimizer for constrained optimization: L. Kyriakidis et al.,” Optim. Eng., pp. 1–52, 2025.

J. Wu, X. Yu, K. He, Z. Gao, and T. Gong, “Promise: A pre-trained knowledge-infused multimodal representation learning framework for medication recommendation,” Inf. Process. Manag., vol. 61, no. 4, p. 103758, 2024.




DOI: https://doi.org/10.47738/jads.v7i1.969

Refbacks

  • There are currently no refbacks.


Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0