A Grid-search Method Approach for Hyperparameter Evaluation and Optimization on Teachable Machine Accuracy: A Case Study of Sample Size Variation

Edwin Ariesto Umbu Malahina, Gregorius Rinduh Iriane, Yohanes Suban Belutowe, Petrus Katemba, Jimi Asmara
Pages 1008-1025
Abstract Views 1082
PDF Downloads 889
Crossref Citations -
DOI https://doi.org/10.47738/jads.v5i3.290

Abstract


This study aims to evaluate the effectiveness of the grid-search method in hyperparameter optimization on Teachable Machine (TM) using a varying number of image samples. The hyperparameters studied include epoch (e), batch size (b), and learning rate (l). A structured grid-search method approach will be applied to test 216 hyperparameter combinations across 6 categories of sample size per class, namely 10, 25, 50, 100, 250, and 500. The results showed that the optimal combination findings were obtained based on variations in the number of samples as follows: 10 samples using e:100, b:256, l:0.001 get an accuracy range of ≥ 90%; for 25 samples using e:500, b:16, l:0.001 get an accuracy range ≥ 97%; for 50 samples using e:100, b:512, l:0.001 get an accuracy range ≥ 88%; for 100 samples using e:500, b:32, l:0.001 get an accuracy range ≥ 88%; for 250 samples using e:50, b:16, l:0.001 get an accuracy range ≥ 92%, and finally 500 samples using e:500, b:256, l:0.001 get an accuracy range ≥ 96% and on average are able to achieve 100% accuracy from the detection test results of the best value performed for each sample variation of the image object. This research provides significant contributions or benefits in finding the optimal hyperparameter configuration, minimizing overfitting, and shortening the search time for TM accuracy in image classification, particularly in human face recognition. The findings support the development of more efficient and accurate TMs and provide practical guidance for finding better hyperparameter optimization using the grid-search method approach. The results of this study have implications for improving the effectiveness and accuracy of TM models and their development in mobile web applications


Keywords


Teachable Machine; Grid Search; Epoch; Batch Size; Learnig Rate

Full Text:

PDF

References


E. A. U. Malahina, M. Saitakela, S. J. Bulan, M. I. J. Lamabelawa, and Y. S. Belutowe, “Teachable Machine: Optimization of Herbal Plant Image Classification Based on Epoch Value, Batch Size and Learning Rate,” Journal of Applied Data Sciences, vol. 5, no. 2, pp. 532–545, May 2024, doi: 10.47738/jads.v5i2.206.

M. Carney et al., “Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification,” in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, New York, NY, USA: ACM, Apr. 2020, pp. 1–8. doi: 10.1145/3334480.3382839.

E. A. U. Malahina, R. P. Hadjon, and F. Y. Bisilisin, “Teachable Machine: Real-Time Attendance of Students Based on Open Source System,” The IJICS (International Journal of Informatics and Computer Science), vol. 6, no. 3, pp. 140–146, Nov. 2022, doi: 10.30865/ijics.v6i3.4928.

D. Agustian, P. P. G. P. Pertama, P. N. Crisnapati, and P. D. Novayanti, “Implementation of Machine Learning Using Google’s Teachable Machine Based on Android,” in 2021 3rd International Conference on Cybernetics and Intelligent System (ICORIS), Makasar: IEEE, Oct. 2021, pp. 1–7. doi: 10.1109/ICORIS52787.2021.9649528.

T. Bagby, K. Rao, and K. C. Sim, “Efficient Implementation of Recurrent Neural Network Transducer in Tensorflow,” in 2018 IEEE Spoken Language Technology Workshop (SLT), Athens: IEEE, Dec. 2018, pp. 506–512. doi: 10.1109/SLT.2018.8639690.

P. Borisagar, Y. Agrawal, and R. Parekh, “Efficient Vehicle Accident Detection System using Tensorflow and Transfer Learning,” in 2018 International Conference on Networking, Embedded and Wireless Systems (ICNEWS), Bangalore: IEEE, Dec. 2018, pp. 1–6. doi: 10.1109/ICNEWS.2018.8903938.

H. Selcuk Nogay and H. Adeli, “Diagnostic of autism spectrum disorder based on structural brain MRI images using, grid search optimization, and convolutional neural networks,” Biomed Signal Process Control, vol. 79, no. 2, pp. 104234–104254, Jan. 2023, doi: 10.1016/j.bspc.2022.104234.

D. M. Belete and M. D. Huchaiah, “Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results,” International Journal of Computers and Applications, vol. 44, no. 9, pp. 875–886, Sep. 2022, doi: 10.1080/1206212X.2021.1974663.

J. J. N. Wong and N. Fadzly, “Development of species recognition models using Google teachable machine on shorebirds and waterbirds,” Journal of Taibah University for Science, vol. 16, no. 1, pp. 1096–1111, Dec. 2022, doi: 10.1080/16583655.2022.2143627.

S. Salim, M. M. A. Jamil, R. Ambar, W. S. W. Zaki, and S. Mohammad, “Learning Rate Optimization for Enhanced Hand Gesture Recognition using Google Teachable Machine,” in 2023 IEEE 13th International Conference on Control System, Computing and Engineering (ICCSCE), Penang: IEEE, Aug. 2023, pp. 332–337. doi: 10.1109/ICCSCE58721.2023.10237148.

M. K. Jha, S. Shukla, A. P. Singh, and V. Shukla, “Advancing Image Classification Through Self-teachable Machine Models and Transfer Learning,” Switzerland: Springer, 2024, pp. 361–373. doi: 10.1007/978-3-031-56700-1_29.

P. Kaur, S. Harnal, V. Gautam, M. P. Singh, and S. P. Singh, “A novel transfer deep learning method for detection and classification of plant leaf disease,” J Ambient Intell Humaniz Comput, vol. 14, no. 9, pp. 12407–12424, Sep. 2023, doi: 10.1007/s12652-022-04331-9.

J. Wang, Z. Zhang, Z. Liu, B. Han, H. Bao, and S. Ji, “Digital twin aided adversarial transfer learning method for domain adaptation fault diagnosis,” Reliab Eng Syst Saf, vol. 234, no. 15, pp. 109152–109166, Jun. 2023, doi: 10.1016/j.ress.2023.109152.

N. Chockwanich and V. Visoottiviseth, “Intrusion Detection by Deep Learning with TensorFlow,” in 2019 21st International Conference on Advanced Communication Technology (ICACT), PyeongChang: IEEE, Feb. 2019, pp. 654–659. doi: 10.23919/ICACT.2019.8701969.

S. Asif, M. Zhao, F. Tang, and Y. Zhu, “An enhanced deep learning method for multi-class brain tumor classification using deep transfer learning,” Multimed Tools Appl, vol. 82, no. 20, pp. 31709–31736, Aug. 2023, doi: 10.1007/s11042-023-14828-w.

F. Lingua, N. C. Coops, and V. C. Griess, “Valuing cultural ecosystem services combining deep learning and benefit transfer approach,” Ecosyst Serv, vol. 58, no. 20, pp. 101487–101489, Dec. 2022, doi: 10.1016/j.ecoser.2022.101487.

P. Liu et al., “A CNN-based transfer learning method for leakage detection of pipeline under multiple working conditions with AE signals,” Process Safety and Environmental Protection, vol. 170, pp. 1161–1172, Feb. 2023, doi: 10.1016/j.psep.2022.12.070.

T. Toivonen, I. Jormanainen, J. Kahila, M. Tedre, T. Valtonen, and H. Vartiainen, “Co-Designing Machine Learning Apps in K–12 With Primary School Children,” in 2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT), Tartu: IEEE, Jul. 2020, pp. 308–310. doi: 10.1109/ICALT49669.2020.00099.

S. I. Nafisah and G. Muhammad, “Tuberculosis detection in chest radiograph using convolutional neural network architecture and explainable artificial intelligence,” Neural Comput Appl, vol. 36, no. 1, pp. 111–131, Jan. 2024, doi: 10.1007/s00521-022-07258-6.

T. L. Kurz, S. Jayasuriya, K. Swisher, J. Mativo, R. Pidaparti, and D. T. Robinson, “The Impact of Teachable Machine on Middle School Teachers’ Perceptions of Science Lessons after Professional Development,” Educ Sci (Basel), vol. 14, no. 4, p. 417 -433, Apr. 2024, doi: 10.3390/educsci14040417.

T. L. Kurz, S. Jayasuriya, K. Swisher, J. Mativo, R. Pidaparti, and D. T. Robinson, “The Impact of Teachable Machine on Middle School Teachers’ Perceptions of Science Lessons after Professional Development,” Educ Sci (Basel), vol. 14, no. 4, pp. 417–433, Apr. 2024, doi: 10.3390/educsci14040417.

S. Mishra, M. Ryerkerk, Y. Lockerman, D. Eis, and J. M. Rzeszotarski, “Teachable Facets: A Framework of Interactive Machine Teaching for Information Filtering,” in Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval, New York, NY, USA: ACM, Mar. 2024, pp. 178–188. doi: 10.1145/3627508.3638289.

S. Shafi and A. Assad, “Exploring the Relationship Between Learning Rate, Batch Size, and Epochs in Deep Learning: An Experimental Study,” in Soft Computing for Problem Solving, Singapore: Springer, 2023, pp. 201–209. doi: 10.1007/978-981-19-6525-8_16.

R. Lin, “Analysis on the Selection of the Appropriate Batch Size in CNN Neural Network,” in 2022 International Conference on Machine Learning and Knowledge Engineering (MLKE), Guilin: IEEE, Feb. 2022, pp. 106–109. doi: 10.1109/MLKE55170.2022.00026.

R. D. Nurfita and G. Ariyanto, “Implementasi Deep Learning berbasis Tensorflow untuk Pengenalan Sidik Jari,” Emitor: Jurnal Teknik Elektro, vol. 18, no. 1, pp. 22–27, Jun. 2018, doi: 10.23917/emitor.v18i01.6236.

Y. Sari, Y. F. Arifin, Novitasari, and M. R. Faisal, “The Effect of Batch Size and Epoch on Performance of ShuffleNet-CNN Architecture for Vegetation Density Classification,” in 7th International Conference on Sustainable Information Engineering and Technology 2022, New York, NY, USA: ACM, Nov. 2022, pp. 39–46. doi: 10.1145/3568231.3568239.

N. Das and S. Das, “Epoch and accuracy based empirical study for cardiac MRI segmentation using deep learning technique,” PeerJ, vol. 11, no. 3, pp. 14939–14953, Mar. 2023, doi: 10.7717/peerj.14939.

N. K. R. Mirayanti, S. Sariyasa, and I. G. A. Gunadi, “Batch size and learning rate effect in covid-19 classification using CNN,” Jurnal Mantik, vol. 7, no. 3, pp. 1752–1765, Nov. 2023, doi: https://doi.org/10.35335/mantik.v7i3.4177.

J. Pan, “The impact of learning rate and data size on CNN for skin cancer detection,” in Second International Conference on Medical Imaging and Additive Manufacturing (ICMIAM 2022), Y. Yusof, Ed., Xiamen: SPIE, Jun. 2022, pp. 28–41. doi: 10.1117/12.2636723.

A. Johny and K. N. Madhusoodanan, “Dynamic Learning Rate in Deep CNN Model for Metastasis Detection and Classification of Histopathology Images,” Comput Math Methods Med, vol. 2021, no. 2, pp. 1–13, Oct. 2021, doi: 10.1155/2021/5557168.

M. Kuehne, L. Polotzek, A. Haghikia, T. Zaehle, and J. S. Lobmaier, “I spy with my little eye: The detection of changes in emotional faces and the influence of facial feedback in Parkinson disease,” Eur J Neurol, vol. 30, no. 3, pp. 622–630, Mar. 2023, doi: 10.1111/ene.15647.

I. Vukovic, P. Cisar, K. Kuk, M. Bandjur, and B. Popovic, “Influence of Image Enhancement Techniques on Effectiveness of Unconstrained Face Detection and Identification,” Elektronika ir Elektrotechnika, vol. 27, no. 5, pp. 49–58, Oct. 2021, doi: 10.5755/j02.eie.29081.

S. Venkatesh, K. Raja, R. Ramachandra, and C. Busch, “On the Influence of Ageing on Face Morph Attacks: Vulnerability and Detection,” in 2020 IEEE International Joint Conference on Biometrics (IJCB), USA: IEEE, Sep. 2020, pp. 1–10. doi: 10.1109/IJCB48548.2020.9304856.

X. Jiang and C. Xu, “Deep Learning and Machine Learning with Grid Search to Predict Later Occurrence of Breast Cancer Metastasis Using Clinical Data,” J Clin Med, vol. 11, no. 19, pp. 5772–5791, Sep. 2022, doi: 10.3390/jcm11195772.




DOI: https://doi.org/10.47738/jads.v5i3.290

Refbacks

  • There are currently no refbacks.


Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0