.png)
Optimizing LSTM with Grid Search and Regularization Techniques to Enhance Accuracy in Human Activity Recognition
Abstract
This study aims to enhance the accuracy of Long Short-Term Memory (LSTM) models for human activity recognition using the UCI Human Activity Recognition (HAR) dataset. The dataset comprises time-series data from accelerometer and gyroscope sensors on smartphones worn by 30 volunteers as they performed everyday activities such as walking, climbing stairs, descending stairs, sitting, standing, and lying down. Optimization was carried out using Grid Search for hyperparameter tuning and L2 regularization to prevent overfitting. The results show that the optimized LSTM model improved accuracy from 92.33% to 94.50%, precision from 93.12% to 94.61%, recall from 92.33% to 94.50%, and F1-score from 92.32% to 94.51% compared to the standard LSTM model. Despite these improvements, the study encountered several challenges, particularly in tuning hyperparameters, which required significant computational resources and time due to the complexity of the search space. Additionally, balancing regularization to prevent both underfitting and overfitting proved to be a delicate process. Further limitations include the model's performance variability with different sensor placements and potential overfitting to specific activity patterns. However, the implementation of hyperparameter optimization and regularization proved effective in improving the model's ability to recognize human activity patterns from complex sensor data. Therefore, this approach holds significant potential for broader applications in sensor-based human activity recognition systems, though further research is needed to address these limitations and generalize the findings.
Keywords
Full Text:
PDFReferences
S. Qiu, “Sensor network oriented human motion capture via wearable intelligent system,” Int. J. Intell. Syst., vol. 37, no. 2, pp. 1646–1673, 2022, doi: 10.1002/int.22689.
J. Zhang, “3D Printable, ultra-stretchable, Self-healable, and self-adhesive dual cross-linked nanocomposite ionogels as ultra-durable strain sensors for motion detection and wearable human-machine interface,” Chem. Eng. J., vol. 431, 2022, doi: 10.1016/j.cej.2021.133949.
L. Tong, “A Novel Deep Learning Bi-GRU-I Model for Real-Time Human Activity Recognition Using Inertial Sensors,” IEEE Sens. J., vol. 22, no. 6, pp. 6164–6174, 2022, doi: 10.1109/JSEN.2022.3148431.
S. Kobayashi, “MarNASNets: Toward CNN Model Architectures Specific to Sensor-Based Human Activity Recognition,” IEEE Sens. J., vol. 23, no. 16, pp. 18708–18717, 2023, doi: 10.1109/JSEN.2023.3292380.
S. Defit, A. P. Windarto, and P. Alkhairi, “Comparative Analysis of Classification Methods in Sentiment Analysis: The Impact of Feature Selection and Ensemble Techniques Optimization,” Telematika, vol. 17, no. 1, pp. 52–67, 2024.
P. Vijayaraghavan, “Hierarchical ensembles of FeCo metal-organic frameworks reinforced nickel foam as an impedimetric sensor for detection of IL-1RA in human samples,” Chem. Eng. J., vol. 458, 2023, doi: 10.1016/j.cej.2023.141444.
Z. Fu, “Anti-freeze hydrogel-based sensors for intelligent wearable human-machine interaction,” Chem. Eng. J., vol. 481, 2024, doi: 10.1016/j.cej.2024.148526.
C. Wei, “Two-dimensional Bi2O2S based high-sensitivity and rapid-response humidity sensor for respiratory monitoring and Human-Machine Interaction,” Chem. Eng. J., vol. 485, 2024, doi: 10.1016/j.cej.2024.149805.
M. F. Yacoub, H. A. Maghawry, N. A. Helal, S. V. Soto, and T. F. Gharib, “An Efficient 2-Stages Classification Model for Students Performance Prediction BT - Proceedings of the 8th International Conference on Advanced Intelligent Systems and Informatics 2022,” A. E. Hassanien, V. Snášel, M. Tang, T.-W. Sung, and K.-C. Chang, Eds., Cham: Springer International Publishing, 2023, pp. 107–122.
R. Yusuf et al., “Application of Analytical Hierarchy Process Method for SQM on Customer Satisfaction,” J. Phys. Conf. Ser., vol. 1783, no. 1, 2021, doi: 10.1088/1742-6596/1783/1/012019.
M. Rane et al., “Breast Cancer Detection Using Machine Learning,” Lect. Notes Networks Syst., vol. 624 LNNS, no. July, pp. 399–406, 2023, doi: 10.1007/978-3-031-25344-7_36.
P. Alkhairi, E. R. Batubara, R. Rosnelly, W. Wanayaumini, and H. S. Tambunan, “Effect of Gradient Descent With Momentum Backpropagation Training Function in Detecting Alphabet Letters,” Sinkron, vol. 8, no. 1, pp. 574–583, 2023, doi: 10.33395/sinkron.v8i1.12183.
P. Alkhairi and A. P. Windarto, “Classification Analysis of Back propagation-Optimized CNN Performance in Image Processing,” J. Syst. Eng. Inf. Technol., vol. 2, no. 1, pp. 8–15, 2023.
S. Y. Xiong, “A Proposed Hybrid CNN-RNN Architecture for Student Performance Prediction,” Int. J. Intell. Syst. Appl. Eng., vol. 10, no. 3, pp. 347–355, 2022.
L. Zhao, “CNN, RNN, or ViT? An Evaluation of Different Deep Learning Architectures for Spatio-Temporal Representation of Sentinel Time Series,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 16, pp. 44–56, 2023, doi: 10.1109/JSTARS.2022.3219816.
A. Dahou, “MLCNNwav: Multilevel Convolutional Neural Network With Wavelet Transformations for Sensor-Based Human Activity Recognition,” IEEE Internet Things J., vol. 11, no. 1, pp. 820–828, 2024, doi: 10.1109/JIOT.2023.3286378.
W. Li, “Multi-characteristic tannic acid-reinforced polyacrylamide/sodium carboxymethyl cellulose ionic hydrogel strain sensor for human-machine interaction,” Int. J. Biol. Macromol., vol. 254, 2024, doi: 10.1016/j.ijbiomac.2023.127434.
J. E. Hyun, “Wearable ion gel based pressure sensor with high sensitivity and ultra-wide sensing range for human motion detection,” Chem. Eng. J., vol. 484, 2024, doi: 10.1016/j.cej.2024.149464.
N. Razali, A. Mustapha, N. Arbaiy, and P. C. Lin, “Deep Learning for Football Outcomes Prediction based on Football Rating System,” AIP Conf. Proc., vol. 2644, no. November, 2022, doi: 10.1063/5.0104587.
S. Chaudhuri et al., “Artificial intelligence enabled applications in kidney disease,” Semin. Dial., vol. 34, no. 1, pp. 5–16, 2021, doi: 10.1111/sdi.12915.
L. Ö. Polatli and M. A. Karadayi, “Sağlık Hizmetlerinde Güncel Makine Öğrenmesi Algoritmaları A Review on Machine Learning Algorithms in Healthcare,” vol. 6, no. 2, pp. 117–143, 2022.
A. P. Windarto, T. Herawan, and P. Alkhairi, “Early Detection of Breast Cancer Based on Patient Symptom Data Using Naive Bayes Algorithm on Genomic Data,” in Artificial Intelligence, Data Science and Applications, Y. Farhaoui, A. Hussain, T. Saba, H. Taherdoost, and A. Verma, Eds., Cham: Springer Nature Switzerland, 2024, pp. 478–484.
A. P. Windarto, I. R. Rahadjeng, M. N. H. Siregar, and P. Alkhairi, “Deep Learning to Extract Animal Images With the U-Net Model on the Use of Pet Images,” J. MEDIA Inform. BUDIDARMA, vol. 8, no. 1, pp. 468–476, 2024.
M. A. Lubis, D. G. S. Saragih, I. D. Anastasia, A. P. Windarto, and P. Alkhairi, “Application of the ANN Algorithm to Predict Access to Drinkable Water in North Sumatra Regency/City,” Int. J. Informatics Data Sci., vol. 1, no. 1, pp. 18–25, 2023.
R. Rahad, “A Novel Plasmonic MIM Sensor Using Integrated 1 × 2 Demultiplexer for Individual Lab-on-Chip Detection of Human Blood Group and Diabetes Level in the Visible to Near-Infrared Region,” IEEE Sens. J., vol. 24, no. 8, pp. 12034–12041, 2024, doi: 10.1109/JSEN.2024.3372692.
A. Saha, “A Survey of Machine Learning and Meta-heuristics Approaches for Sensor-based Human Activity Recognition Systems,” J. Ambient Intell. Humaniz. Comput., vol. 15, no. 1, pp. 29–56, 2024, doi: 10.1007/s12652-022-03870-5.
A. Ferrari, “Deep learning and model personalization in sensor-based human activity recognition,” J. Reliab. Intell. Environ., vol. 9, no. 1, pp. 27–39, 2023, doi: 10.1007/s40860-021-00167-w.
T. Zhu, “Multifunctional hydrophobic fabric-based strain sensor for human motion detection and personal thermal management,” J. Mater. Sci. Technol., vol. 138, pp. 108–116, 2023, doi: 10.1016/j.jmst.2022.08.010.
H. Liang, “Wearable and Multifunctional Self-Mixing Microfiber Sensor for Human Health Monitoring,” IEEE Sens. J., vol. 23, no. 3, pp. 2122–2127, 2023, doi: 10.1109/JSEN.2022.3225196.
X. Wang, “Wearable Sensors-Based Hand Gesture Recognition for Human-Robot Collaboration in Construction,” IEEE Sens. J., vol. 23, no. 1, pp. 495–505, 2023, doi: 10.1109/JSEN.2022.3222801.
M. A. Khatun, “Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor,” IEEE J. Transl. Eng. Heal. Med., vol. 10, 2022, doi: 10.1109/JTEHM.2022.3177710.
B. B. Yousif, M. M. Ata, N. Fawzy, and M. Obaya, “Toward an Optimized Neutrosophic k-Means with Genetic Algorithm for Automatic Vehicle License Plate Recognition (ONKM-AVLPR),” IEEE Access, vol. 8, pp. 49285–49312, 2020, doi: 10.1109/ACCESS.2020.2979185.
M. Yaqub, J. Feng, M. S. Zia, K. Arshid, and K. Jia, “brain sciences State-of-the-Art CNN Optimizer for Brain Tumor Segmentation in Magnetic Resonance Images”, doi: 10.3390/brainsci10070427.
A. Kumar, “Bitcoin Price Prediction Using Sentiment Analysis and Long Short-Term Memory (LSTM),” Int. J. Intell. Syst. Appl. Eng., vol. 11, no. 7, pp. 480–485, 2023.
V. Yadav, “Long short term memory (LSTM) model for sentiment analysis in social data for e-commerce products reviews in Hindi languages,” Int. J. Inf. Technol., vol. 15, no. 2, pp. 759–772, 2023, doi: 10.1007/s41870-022-01010-y.
M. Ma, “Predicting machine’s performance record using the stacked long short-term memory (LSTM) neural networks,” J. Appl. Clin. Med. Phys., vol. 23, no. 3, 2022, doi: 10.1002/acm2.13558.
J. Liu, “Prediction of nucleosome dynamic interval based on long-short-term memory network (LSTM),” J. Bioinform. Comput. Biol., vol. 20, no. 3, 2022, doi: 10.1142/S0219720022500093.
R. Huang, “Well performance prediction based on Long Short-Term Memory (LSTM) neural network,” J. Pet. Sci. Eng., vol. 208, 2022, doi: 10.1016/j.petrol.2021.109686.
Y. Kurniawati, “Optimization of Backpropagation Using Harmony Search for Gold Price Forecasting,” Pakistan J. Stat. Oper. Res., vol. 18, no. 3, pp. 589–599, 2022, doi: 10.18187/pjsor.v18i3.3915.
T. Yuan, W. Liu, J. Han, and F. Lombardi, “High Performance CNN Accelerators Based on Hardware and Algorithm Co-Optimization,” IEEE Trans. Circuits Syst. I Regul. Pap., vol. 68, no. 1, pp. 250–263, 2021, doi: 10.1109/TCSI.2020.3030663.
A. P. Windarto, T. Herawan, and P. Alkhairi, “Prediction of Kidney Disease Progression Using K-Means Algorithm Approach on Histopathology Data,” in Artificial Intelligence, Data Science and Applications, Y. Farhaoui, A. Hussain, T. Saba, H. Taherdoost, and A. Verma, Eds., Cham: Springer Nature Switzerland, 2024, pp. 492–497.
P. Alkhairi, W. Wanayumini, and B. H. Hayadi, “Analysis of the adaptive learning rate and momentum effects on prediction problems in increasing the training time of the backpropagation algorithm,” AIP Conf. Proc., vol. 3048, no. 1, p. 20049, 2024, doi: 10.1063/5.0203374.
K. S. Chong, “Comparison of Naive Bayes and SVM Classification in Grid-Search Hyperparameter Tuned and Non-Hyperparameter Tuned Healthcare Stock Market Sentiment Analysis,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 12, pp. 90–94, 2022, doi: 10.14569/IJACSA.2022.0131213.
Z. Ragala, A. Retbi, and S. Bennani, “Overview of Gradient Descent Algorithms: Application to Railway Regularity BT - Proceedings of the 8th International Conference on Advanced Intelligent Systems and Informatics 2022,” A. E. Hassanien, V. Snášel, M. Tang, T.-W. Sung, and K.-C. Chang, Eds., Cham: Springer International Publishing, 2023, pp. 39–49.
D. L. M, “An Improved Convolution Neural Network and Modified Regularized K-Means-Based Automatic Lung Nodule Detection and Classification,” J. Digit. Imaging, vol. 36, no. 4, pp. 1431–1446, 2023, doi: 10.1007/s10278-023-00809-w.
S. Dodia, “A novel receptive field-regularized V-net and nodule classification network for lung nodule detection,” Int. J. Imaging Syst. Technol., vol. 32, no. 1, pp. 88–101, 2022, doi: 10.1002/ima.22636.
DOI: https://doi.org/10.47738/jads.v5i4.433
Refbacks
- There are currently no refbacks.
Journal of Applied Data Sciences
| ISSN | : | 2723-6471 (Online) |
| Publisher | : | Bright Publisher |
| Website | : | http://bright-journal.org/JADS |
| : | taqwa@amikompurwokerto.ac.id (principal contact) | |
| support@bright-journal.org (technical issues) |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0