Multi-Level Stacked Ensemble Learning with Multi-Search Hyperparameter Optimization for Stock Price Forecasting

Abstract

Stock price forecasting remains a challenging task due to the nonlinear, dynamic, and highly volatile nature of financial markets. Traditional forecasting methods often struggle to capture complex relationships among market variables, resulting in limited predictive accuracy. This study proposes a Multi Level Stacked Ensemble Learning (MML-SEL) framework integrated with multiple hyperparameter optimization techniques to improve stock price forecasting performance. The proposed architecture combines heterogeneous machine learning models across several hierarchical stacking layers, including Random Forest, Stochastic Gradient Descent, Multilayer Perceptron, Long Short-Term Memory, eXtreme Gradient Boosting, Gradient Boosting, and Support Vector Regression. To further enhance model performance, three hyperparameter optimization methods—Grid Search, Random Search, and Bayes Search—are employed and compared. Historical stock market data consisting of Open, High, Low, Close, and Volume variables were collected from three different stock exchanges, represented by TLKM (Indonesia Stock Exchange), MSFT (NASDAQ), and BIDU (Hong Kong Stock Exchange). The dataset was chronologically divided into training and testing sets, followed by data preprocessing, feature scaling, model training, stacking integration, and performance evaluation. Forecasting accuracy was assessed using R-Squared (R²), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE). The experimental results demonstrate that the proposed MML-SEL framework achieves strong predictive performance across all datasets. The best results were obtained using Grid Search optimization, producing an R² value of 0.9996, MAE of 0.0114, and RMSE of 0.0202 on the MSFT dataset. The findings indicate that hierarchical ensemble learning combined with systematic hyperparameter optimization can significantly improve forecasting accuracy and model stability. This study contributes a scalable and effective forecasting framework that can support data-driven investment analysis and financial decision-making in diverse stock market environments.