Application of Convolutional Neural Networks for Automated Iris Edge Detection in Sleepiness Monitoring during Blended Learning

Tukino Tukino, Yuhandri Yuhandri, Sumijan Sumijan

Abstract


This study introduces a novel lightweight Convolutional Neural Network (CNN) model, T-Net, designed for real-time drowsiness detection based on eye closure patterns. The model was developed to address the prevalent issue of student fatigue in resource-constrained environments, such as during prolonged online learning or blended learning sessions. Unlike traditional deep learning models, T-Net prioritizes efficiency while maintaining high accuracy, making it suitable for deployment on devices with limited computational resources. The model uses a 68-point facial landmark detection technique to extract the eye region and accurately classify eyelid states (open or closed). Evaluated on two benchmark datasets, Dataset-1 (342 eye images) and Dataset-2 (1,510 eye images), T-Net demonstrated superior performance, achieving classification accuracies of 99.33% and 99.27%, respectively, outperforming other pre-trained models such as VGG19, ResNet50, and MobileNetV2. Usability testing revealed a high acceptance rate, with a System Usability Scale (SUS) score of 84.5, indicating the system’s practicality for real-world use. Additionally, statistical analysis showed a significant correlation (r = 0.67, p < 0.01) between prolonged screen time and the emergence of visual fatigue symptoms. This study highlights the effectiveness of a lightweight CNN approach for real-time fatigue monitoring, offering a balance between performance and computational efficiency. The results suggest that T-Net can be effectively integrated into student monitoring systems to ensure alertness during learning sessions. Future research will focus on expanding the dataset, integrating infrared imaging for low-light environments, and incorporating additional fatigue indicators such as yawning and head pose.


Keywords


Convolutional Neural Networks; Iris Edge Detection; Sleepiness Monitoring; Blended Learning; Computer Vision in Education

Full Text:

PDF

References


H. Summala, "Towards understanding motivational and emotional factors in student behaviour: Comfort through satisficing," in Modelling Student Behaviour in Automotive Environments: Critical Issues in Student Interactions With Intelligent Transport Systems, London, U.K.: Springer, 2021, pp. 189–207. [Online]. Available: https://doi.org/10.1007/978-1-84628-618-6_11

. T. Igasaki, K. Nagasawa, N. Murayama, and Z. Hu, "Drowsiness estimation under driving environment by heart rate variability and/or breathing rate variability with logistic regression analysis," in Proc. 8th Int. Conf. Biomed. Eng. Informat. (BMEI), Oct. 2023, pp. 189–193. [Online]. Available: https://doi.org/10.1109/BMEI.2015.7401530

. M. Papakostas, K. Das, M. Abouelenien, R. Mihalcea, and M. Burzo, "Distracted and drowsy driving modeling using deep physiological representations and multitask learning," Appl. Sci., vol. 11, no. 1, p. 88, Dec. 2020. [Online]. Available: https://doi.org/10.3390/app11010088

. M. Awais, N. Badruddin, and M. Drieberg, "A hybrid approach to detect student drowsiness utilizing physiological signals to improve system performance and wearability," Sensors, vol. 17, no. 9, p. 1991, Aug. 2022. [Online]. Available: https://doi.org/10.3390/s17091991

. M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas, and A. Mahmood, "A survey on state-of-the-art drowsiness detection techniques," IEEE Access, vol. 7, pp. 61904–61919, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2914985

. Zebra, "Drowsy Driving Statistics," The Zebra, Jan. 2023. [Online]. Available: https://www.thezebra.com/resources/research/drowsy-driving-statistics/

. M. M. Ahsan, Y. Li, J. Zhang, M. T. Ahad, dan M. M. S. Yazdan, "Face recognition in an unconstrained and real-time environment using novel BMC-LBPH methods incorporates with DJI vision sensor," J. Sensor Actuator Netw., vol. 9, no. 4, p. 54, Nov. 2020. [Online]. https://doi.org/10.3390/jsan9040054

. A. Liu, Z. Li, L. Wang, dan Y. Zhao, "A practical student fatigue detection algorithm based on eye state," dalam Proc. Asia–Pacific Conf. Postgraduate Res. Microelectron. Electron. (PrimeAsia), Sep. 2020, hlm. 235–238.

. A. Królak dan P. Strumiłło, "Eye-blink detection system for human–computer interaction," Universal Access Inf. Soc., vol.11, no. 4, hlm. 409–419, Nov. 2022. [Online]. https://doi.org/10.1007/s10209-011-0256-6SpringerLink+1SpringerLink+1

. Z.-T. Liu et al., "Eye state detection based on weight binarization convolution neural network and transfer learning," Appl. Soft Comput., vol. 109, Sep. 2021, Art. no. 107565. [Online]. https://doi.org/10.1016/j.asoc.2021.107565

. S. Fuangkaew dan K. Patanukhom, "Eye state detection and eye sequence classification for paralyzed patient interaction," dalam Proc. 2nd IAPR Asian Conf. Pattern Recognit., Nov. 2023, hlm. 376–380.

. M. A. R. Ahad, S. Kobashi, dan J. M. R. Tavares, "Advancements of image processing and vision in healthcare," J. Healthcare Eng., vol. 2019, Art. no. 8458024. [Online]. Available: https://doi.org/10.1155/2019/8458024

. A. Koesdwiady, R. Soua, F. Karray, dan M. S. Kamel, "Recent trends in student safety monitoring systems: State of the art and challenges," IEEE Trans. Veh. Technol., vol. 66, no. 6, pp. 4550–4563, Jun. 2022. [Online]. Available: https://doi.org/10.1109/TVT.2016.2582740

. A.-C. Phan, T.-N. Trieu, dan T.-C. Phan, "Student drowsiness detection and smart alerting using deep learning and IoT," Internet Things, vol. 22, Art. no. 100705, Jul. 2023. [Online]. Available: https://doi.org/10.1016/j.iot.2023.100705

. T. Faisal et al., "Systematic development of real-time student drowsiness detection system using deep learning," IAES Int. J. Artif. Intell. (IJ-AI), vol. 11, no. 1, p. 148, Mar. 2022. [Online]. Available: https://doi.org/10.11591/ijai.v11.i1.pp148-155

. B. Ganguly, D. Dey, dan S. Munshi, "An integrated system for students’ drowsiness detection using deep learning frameworks," dalam Proc. IEEE VLSI Device Circuit Syst. (VLSI DCS), Feb. 2022, pp. 55–59.

. E. M. Lopez et al., "Student drowsiness detection by applying deep learning techniques to sequences of images," Appl. Sci., vol. 12, no. 3, p. 1145, 2022. [Online]. Available: https://doi.org/10.3390/app12031145

. R. Florez et al., "A CNN-based approach for student drowsiness detection by real-time eye state identification," Appl. Sci., vol. 13, no. 13, p. 7849, Jul. 2023. [Online]. Available: https://doi.org/10.3390/app13137849

. I. Jahan et al., "4D: A real-time student drowsiness detector using deep learning," Electronics, vol. 12, no. 1, p. 235, Jan. 2023. [Online]. Available: https://doi.org/10.3390/electronics12010235

. B. Akrout dan S. Fakhfakh, "How to prevent students before their sleepiness using deep learning-based approach," Electronics, vol. 12, no. 4, p. 965, Feb. 2023. [Online]. Available: https://doi.org/10.3390/electronics12040965

. S. Dewangan dan P. K. Bora, "Lightweight convolutional neural network architecture for real-time student drowsiness detection," Neural Comput. Appl., vol. 35, no. 10, pp. 6549–6564, 2023. [Online]. Available: https://doi.org/10.1007/s00521-022-07608-2

. R. Singh, H. Kaur, dan S. Jain, "Student drowsiness detection system based on image processing and machine learning," dalam Proc. Int. Conf. Comput. Commun. Autom. (ICCCA), 2022, pp. 1–6.

. H. Dutta, A. J. Mashruwala, dan P. M. Mahalle, "Detection of student drowsiness using eye-blinking behavior," dalam Proc. Int. Conf. Comput. Sustain. Glob. Dev. (INDIACom), 2021, pp. 327–331.

. H. Liu, Z. Lin, Y. Jin, dan Z. Zhang, "Student drowsiness detection based on yawning analysis," IEEE Access, vol. 8, pp. 38400–38411, 2020. [Online]. Available: https://doi.org/10.1109/ACCESS.2020.2975303

. A. E. Hassan dan A. Z. S. Saif, "Student drowsiness detection system based on facial features and biological signals," Appl. Sci., vol. 11, no. 17, p. 7923, Aug. 2021. [Online]. Available: https://doi.org/10.3390/app11177923

. S. C. Mukhopadhyay, "Wearable sensors for human activity monitoring: A review," IEEE Sensors J., vol. 15, no. 3, pp. 1321–1330, Mar. 2023. [Online]. Available: https://doi.org/10.1109/JSEN.2014.2370945

. Y. Weng, S. Wu, Y. Gao, dan L. Chen, "Student drowsiness detection via a multi-channel second-order pooling convolutional neural network," IEEE Trans. Cogn. Develop. Syst., vol. 14, no. 2, pp. 556–566, Jun. 2022. [Online]. Available: https://doi.org/10.1109/TCDS.2021.3109293

. M. Q. Raza, J. Iqbal, dan A. R. Baig, "Student drowsiness detection through facial feature recognition," dalam Proc. Int. Conf. Innovative Comput. Commun. (ICICC), 2022, pp. 235–245.

. J. L. Atienza dan A. Canedo, "Student drowsiness detection using infrared camera and artificial neural networks," Sensors, vol. 22, no. 1, p. 289, Dec. 2021. [Online]. Available: https://doi.org/10.3390/s22010289

. M. Z. Islam, M. S. Hossain, dan G. Muhammad, "Explainable and lightweight ensemble learning model for student drowsiness detection," IEEE Trans. Intell. Transp. Syst., vol. 23, no. 6, pp. 5095–5104, Jun. 2022. [Online]. Available: https://doi.org/10.1109/TITS.2021.3075591

. Y. Cui et al., "Real-time student drowsiness detection based on transfer learning using a hybrid model," Appl. Sci., vol. 11, no. 12, p. 5402, 2021. [Online]. Available: https://doi.org/10.3390/app11125402

. M. Rana, T. Dutta, dan K. Nath, "Student drowsiness detection based on eye blink duration using deep learning," dalam Proc. 3rd Int. Conf. Comput. Commun. (IC4), 2022, pp. 80–84.

. W. Wang, S. An, dan L. Chen, "Real-time student drowsiness detection using facial landmarks and recurrent neural networks," IEEE Access, vol. 9, pp. 124327–124337, 2021. [Online]. Available: https://doi.org/10.1109/ACCESS.2021.3109752

. N. Dalal dan B. Triggs, "Histograms of oriented gradients for human detection," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2022, pp. 886–893.

. K. Simonyan dan A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint, arXiv:1409.1556, Sep. 2024. [Online]. Available: https://arxiv.org/abs/1409.1556

. C. Szegedy et al., "Going deeper with convolutions," dalam Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 1–9.

. S. Hochreiter dan J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, no. 8, pp. 1735–1780, 2022. [Online]. Available: https://doi.org/10.1162/neco.1997.9.8.1735

. M. Abadi et al., "TensorFlow: A system for large-scale machine learning," Proc. 12th USENIX Symp. Oper. Syst. Design Implement. (OSDI), 2022, pp. 265–283. [Online]. Available:https://www.usenix.org/conference/osdi16/technical-sessions/ presentation/ abadi

. F. Chollet et al., "Keras," 2023. [Online]. Available: https://keras.io/

. S. Ioffe dan C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," dalam Proc. 32nd Int. Conf. Mach. Learn. (ICML), 2023, pp. 448–456. [Online]. Available: https://proceedings.mlr.press/v37/ioffe15.html




DOI: https://doi.org/10.47738/jads.v6i4.882

Refbacks

  • There are currently no refbacks.



Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0