Multiple Choice Question Difficulty Level Classification with Multi Class Confusion Matrix in the Online Question Bank of Education Gallery

Abstract

The importance of test question planning as a critical element in improving the quality of education is undeniable as it helps teachers evaluate student understanding. The creation of questions must consider the level of difficulty, which is often divided into three categories: easy, medium, and difficult. Predicting the difficulty level of questions has great importance as it helps teachers create test questions that match students' abilities. In this study, we view the identification of item difficulty as a classification problem. The data used includes questions from elementary and junior high school, with various machine learning methods applied to perform classification. We tested Random Forest, Logistic Regression, SVM, Gaussian, and Dense NN, considering embedding, lexical, and syntactic features. The evaluation results show that the best method in identifying the difficulty level of questions in subjects is using Random Forest, resulting in an accuracy of 84%. Meanwhile, in other cases, the best method is also Random Forest, with an accuracy of 80%. Our research shows that the use of feature embedding and TF-IDF has a significant positive impact on the accuracy of the resulting model.