.png)
Improving MCDM University Rankings through Statistical Validation Using Spearman’s Correlation and THE Benchmark
Abstract
The evaluation of higher education institutions is a critical field for informing data-driven policy and institutional benchmarking. A key problem in this area is the lack of transparency and consistency in university rankings, particularly when using Multi-Criteria Decision-Making (MCDM) methods such as MABAC and MAIRCA, with limited research on how weighting techniques affect the reliability and alignment of these rankings with international standards like the Times Higher Education (THE) Rankings. This study proposes the use of MABAC and MAIRCA methods combined with two weighting techniques—Rank Order Centroid (ROC) and Rank Sum (RS)—to assess 20 top Indonesian universities based on five performance indicators: research quality, research environment, teaching, industry, and international outlook. Spearman’s rank correlation is used to compare the MCDM-generated rankings with THE Rankings 2025. The study contributes empirical evidence on the impact of weighting schemes on the consistency and reliability of university rankings and demonstrates that the MAIRCA-ROC method achieves the highest agreement with THE Rankings, with a correlation coefficient of 0.8135 and a p-value of 0.00001. These results validate the use of MCDM methods in higher education evaluation and emphasize the importance of selecting appropriate weighting techniques to develop transparent and robust ranking frameworks that support evidence-based policy decisions.
Keywords
Full Text:
PDFReferences
QS Quacquarelli Symonds, “QS World University Rankings Methodology.” Accessed: Apr. 18, 2025. [Online]. Available: https://support.qs.com/hc/en-gb/articles/4405955370898-QS-World-University-Rankings
Times Higher Education, “World University Rankings 2025: Methodology.” Accessed: Apr. 18, 2025. [Online]. Available: https://www.timeshighereducation.com/world-university-rankings/asia-university-rankings-2025-methodology
G. T. Papanikos, “Times Higher Education University Rankings: Top 100 Best Universities by Country (2011-2025).” [Online]. Available: https://www.atiner.gr/education
I. Z. Mukhametzyanov, “Elimination of the Domains’ Displacement of the Normalized Values in MCDM Tasks: The IZ-Method,” Int J Inf Technol Decis Mak, vol. 23, no. 01, pp. 289–326, Dec. 2022, doi: 10.1142/S0219622023500037.
Z. N. Shehab, R. M. Faisal, and S. W. Ahmed, “Multi-criteria Decision Making (MCDM) Approach for Identifying Optimal Solar Farm Locations: A Multi-Technique Comparative Analysis,” Renew Energy, vol. 237, p. 121787, 2024, doi: https://doi.org/10.1016/j.renene.2024.121787.
S. H. Mian, E. Abouel Nasr, K. Moiduddin, M. Saleh, M. H. Abidi, and H. Alkhalefah, “Assessment of Consolidative Multi-Criteria Decision Making (C-MCDM) Algorithms for Optimal Mapping of Polymer Materials in Additive Manufacturing: A Case Study of Orthotic Application,” Heliyon, vol. 10, no. 10, May 2024, doi: 10.1016/j.heliyon.2024.e30867.
G. Anand, S. Sardar, S. Sah, A. Guha, and D. Das, “Multi-Objective Optimization to Enhance Surface Integrity in WEDM for Al-Matrix Composite: A Comparative Assessment of Self-Weight Adjusting MCDMs and Objective Weight Integrated Hybrid TOPSIS Methods,” Results in Surfaces and Interfaces, vol. 18, Jan. 2025, doi: 10.1016/j.rsurfi.2025.100467.
P. Villalba, A. J. Sánchez-Garrido, and V. Yepes, “A Review of Multi-Criteria Decision-Making Methods for Building Assessment, Selection, and Retrofit,” Journal of Civil Engineering and Management, vol. 30, no. 5, pp. 465–480, May 2024, doi: 10.3846/jcem.2024.21621.
H. Regragui, N. Sefiani, H. Azzouzi, and N. Cheikhrouhou, “A Hybrid Multi-Criteria Decision-Making Approach for Hospitals’ Sustainability Performance Evaluation Under Fuzzy Environment,” International Journal of Productivity and Performance Management, vol. 73, no. 3, pp. 855–888, Mar. 2024, doi: 10.1108/IJPPM-10-2022-0538.
A. Hariri, P. Domingues, and P. Sampaio, “Integration of Multi-Criteria Decision-Making Approaches Adapted for Quality Function Deployment: An Analytical Literature Review and Future Research Agenda,” Nov. 08, 2023, Emerald Publishing. doi: 10.1108/IJQRM-02-2022-0058.
A. Siva Bhaskar and A. Khan, “Comparative Analysis of Hybrid MCDM Methods in Material Selection for Dental Applications,” Expert Syst Appl, vol. 209, p. 118268, Dec. 2022, doi: https://doi.org/10.1016/j.eswa.2022.118268.
N. B. and J. Roy, “A New Hybrid MCDM Framework for Third-Party Logistics Provider Selection under Sustainability Perspectives,” Expert Syst Appl, vol. 234, p. 121009, Dec. 2023, doi: https://doi.org/10.1016/j.eswa.2023.121009.
M. Y. Firdaus and S. Andryana, “Employee Ranking Based on Work Performance Using AHP and VIKOR Methods,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), IEEE, Feb. 2023, pp. 968–973. doi: 10.1109/ICCoSITE57641.2023.10127814.
A. R. Mishra, P. Rani, F. Cavallaro, and A. F. Alrasheedi, “Assessment of Sustainable Wastewater Treatment Technologies Using Interval-Valued Intuitionistic Fuzzy Distance Measure-Based MAIRCA Method,” Facta Universitatis, Series: Mechanical Engineering, vol. 21, no. 3, pp. 359–386, 2023, doi: 10.22190/FUME230901034M.
A. Negi et al., “A Hybrid CRITIC-MAIRCA Framework for Optimal Phase Change Material Selection in Solar Distillation Systems,” International Journal of Thermofluids, vol. 27, May 2025, doi: 10.1016/j.ijft.2025.101167.
Ö. Işık, A. Çalık, and M. Shabir, “A Consolidated MCDM Framework for Overall Performance Assessment of Listed Insurance Companies Based on Ranking Strategies,” Comput Econ, vol. 65, pp. 271–312, Jan. 2024, doi: 10.1007/s10614-024-10578-5.
D. T. Do and N. T. Nguyen, “Applying CoCoSo, MABAC, MAIRCA, EAMR, TOPSIS and Weight Determination Methods for Multi-Criteria Decision Making in Hole Turning Process,” Strojnicky Casopis, vol. 72, no. 2, pp. 15–40, Nov. 2022, doi: 10.2478/scjme-2022-0014.
S. Eti et al., “Assessment of Technical and Financial Challenges for Renewable Energy Project Alternatives,” Clean Eng Technol, vol. 18, Feb. 2024, doi: 10.1016/j.clet.2023.100719.
S. Qahtan et al., “Evaluation of Agriculture-Food 4.0 Supply Chain Approaches using Fermatean Probabilistic Hesitant-Fuzzy Sets Based Decision Making Model,” Appl Soft Comput, vol. 138, May 2023, doi: 10.1016/j.asoc.2023.110170.
M. Zhao, G. Wei, X. Chen, and Y. Wei, “Intuitionistic Fuzzy MABAC Method Based on Cumulative Prospect Theory for Multiple Attribute Group Decision Making,” International Journal of Intelligent Systems, vol. 36, no. 11, pp. 6337–6359, Nov. 2021, doi: 10.1002/int.22552.
C. Jana, H. Garg, M. Pal, B. Sarkar, and G. Wei, “MABAC Framework for Logarithmic Bipolar Fuzzy Multiple Attribute Group Decision-Making for Supplier Selection,” Complex and Intelligent Systems, vol. 10, no. 1, pp. 273–288, Feb. 2024, doi: 10.1007/s40747-023-01108-1.
A. R. Mishra, A. Saha, P. Rani, D. Pamucar, D. Dutta, and I. M. Hezam, “Sustainable Supplier Selection Using HF-DEA-FOCUM-MABAC Technique: A Case Study in the Auto-Making Industry,” Soft comput, vol. 26, no. 17, pp. 8821–8840, Sep. 2022, doi: 10.1007/s00500-022-07192-8.
D. Liu, L. Qiao, C. Liu, B. Liu, and S. Liu, “Evaluation of Urban Quality Improvement Based on the MABAC Method and VIKOR Method: A Case Study of Shandong Province, China,” Sustainability (Switzerland) , vol. 16, no. 8, p. 1, Apr. 2024, doi: 10.3390/su16083308.
H. A. Le, X. T. Hoang, Q. H. Trieu, D. L. Pham, and X. H. Le, “Determining the Best Dressing Parameters for External Cylindrical Grinding Using MABAC Method,” Applied Sciences (Switzerland), vol. 12, no. 16, Aug. 2022, doi: 10.3390/app12168287.
T. van Dua, D. van Duc, N. C. Bao, and D. D. Trung, “Integration of Objective Weighting Methods for Criteria and MCDM Methods: Application in Material Selection,” EUREKA, Physics and Engineering, vol. 2024, no. 2, pp. 131–148, 2024, doi: 10.21303/2461-4262.2024.003171.
P. Sureeyatanapas and K. Damapong, “Performance Assessment and Comparison of Online Food Delivery Service Providers Based upon the Aggregated Perspectives of Restaurant Operators,” Expert Syst Appl, vol. 236, Feb. 2024, doi: 10.1016/j.eswa.2023.121262.
K. Ponhan and P. Sureeyatanapas, “A Comparison Between Subjective and Objective Weighting Approaches for Multi-Criteria Decision Making: A Case of Industrial Location Selection,” Engineering and Applied Science Research, vol. 49, no. 6, pp. 763–771, Dec. 2022, doi: 10.14456/easr.2022.74.
N. Wiangkam, T. Jamrus, and P. Sureeyatanapas, “The Decision-Making for Selecting Cold Chain Logistics Providers in the Food Industry,” Engineering and Applied Science Research, vol. 49, no. 6, pp. 811–818, 2022, doi: 10.14456/easr.2022.79.
R. C. Burk and R. M. Nehring, “An Empirical Comparison of Rank-Based Surrogate Weights in Additive Multiattribute Decision Analysis,” Decision Analysis, vol. 20, no. 1, pp. 55–72, 2023, doi: 10.1287/deca.2022.0456.
M. El Abassi et al., “The Combination of Multi-Criteria Decision-Making (MCDM) and Morphometric Parameters for Prioritizing the Erodibility of Sub-Watersheds in the Ouljet Es Soltane Basin (North of Morocco),” Heliyon, vol. 10, no. 19, pp. 1–27, Oct. 2024, doi: 10.1016/j.heliyon.2024.e38228.
P. A. Ejegwa, N. Kausar, N. Aydin, Y. Feng, and O. A. Olanrewaju, “A New Fermatean Fuzzy Spearman-Like Correlation Coefficient and Its Application in Evaluating Insecurity Problem via Multi-Criteria Decision-Making Approach,” Heliyon, vol. 10, no. 22, Nov. 2024, doi: 10.1016/j.heliyon.2024.e40403.
A. A. Nasser, A. S. A. Alghawli, S. Saleh, and A. A. K. Elsayed, “Health Security Disparities in the Eastern Mediterranean Region: A Comparative Analysis Using an Integrated MCDM and Clustering Approach,” J Biosaf Biosecur, vol. 7, no. 1, pp. 38–51, Mar. 2025, doi: 10.1016/j.jobb.2025.01.001.
M. M. Niazmandi, R. Sedaeesoula, S. Lari, and M. Yousefi, “Integrated Project Delivery (IPD) Capabilities on Reducing Claims in Urban Underground Projects: A Hybrid FAHP-FTOPSIS Approach,” Sustainable Futures, vol. 7, Jun. 2024, doi: 10.1016/j.sftr.2024.100175.
D. T. Do, “Application of FUCA Method for Multi-Criteria Decision Making in Mechanical Machining Processes,” Operational Research in Engineering Sciences: Theory and Applications, vol. 5, no. 3, pp. 131–152, 2022, doi: 10.31181/oresta051022061d.
DOI: https://doi.org/10.47738/jads.v6i3.796
Refbacks
- There are currently no refbacks.
Journal of Applied Data Sciences
| ISSN | : | 2723-6471 (Online) |
| Publisher | : | Bright Publisher |
| Website | : | http://bright-journal.org/JADS |
| : | taqwa@amikompurwokerto.ac.id (principal contact) | |
| support@bright-journal.org (technical issues) |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0