Early Stopping on CNN-LSTM Development to Improve Classification Performance

Abstract

Currently, CNN-LSTM has been widely developed through changes in its architecture and other modifications to improve the performance of this hybrid model. However, some studies pay less attention to overfitting, even though overfitting must be prevented as it can provide good accuracy initially but leads to classification errors when new data is added. Therefore, extra prevention measures are necessary to avoid overfitting. This research uses dropout with early stopping to prevent overfitting. The dataset used for testing is sourced from Twitter; this research also develops architectures using activation functions within each architecture. The developed architecture consists of CNN, MaxPooling1D, Dropout, LSTM, Dense, Dropout, Dense, and SoftMax as the output. Architecture A uses default activations such as ReLU for CNN and Tanh for LSTM. In Architecture B, all activations are replaced by Tanh, and in Architecture C, they are entirely replaced by ReLU. This research also performed hyperparameter tuning such as the number of layers, batch size, and learning rate. This study found that dropout and early stopping can increase accuracy to 85% and prevent overfitting. The best architecture entirely uses ReLU activation as it demonstrates advantages in computational efficiency, convergence speed, the ability to capture relevant patterns, and resistance to noise.