Hyperparameter Tuning in Machine Learning: A Comprehensive Review
Justus A Ilemobayo, Olamide Durodola, Oreoluwa Alade, Opeyemi J Awotunde, Adewumi T Olanrewaju, Olumide Falana, Adedolapo Ogungbire, Abraham Osinuga, Dabira Ogunbiyi, Ark Ifeanyi, Ikenna E Odezuligbo, Oluwagbotemi E Edu
Journal of Engineering Research and Reports · pp. 388–395 · Published 7 Jun 2024
10.9734/jerr/2024/v26i61188Abstract
Hyperparameter tuning is essential for optimizing the performance and generalization of machine learning (ML) models. This review explores the critical role of hyperparameter tuning in ML, detailing its importance, applications, and various optimization techniques. Key factors influencing ML performance, such as data quality, algorithm selection, and model complexity, are discussed, along with the impact of hyperparameters like learning rate and batch size on model training. Various tuning methods are examined, including grid search, random search, Bayesian optimization, and meta-learning. Special focus is given to the learning rate in deep learning, highlighting strategies for its optimization. Trade-offs in hyperparameter tuning, such as balancing computational cost and performance gain, are also addressed. Concluding with challenges and future directions, this review provides a comprehensive resource for improving the effectiveness and efficiency of ML models.
Cited by 213
Hong-Kwan Yoo, Yan Linn Aung, Woo-Seong Che · Applied Sciences · 2025
Yuxin Gao, Baosheng Qin, Fenglian Chen · 2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · 2025
Freddy Mora-Poblete, Javiera Valenzuela-Herrera, Matías Balach · Frontiers in Plant Science · 2026
Francisco García-Varela, Miguel Nussbaum, Marcelo Mendoza · Education Sciences · 2025
Ashagrie Anteneh, Muluken Belachew, Sefefe Birhanu · BMC Public Health · 2025
Kgosietsile D. Mokone, Siphesihle P. Sithungu, Elizabeth M. Ehlers · Proceedings of the 2025 8th International Conference on Computational Intelligence and Intelligent Systems · 2025
Tuong Vi Tran, Aaron Peche, Robert Kringel · Water · 2025
Joowon Oh, Jeaho Lee · Applied Sciences · 2025
Nima Sakhaee, Stephanie Sarrouf, Paul Kim · Journal of Environmental Management · 2025
Mohammad Hossein Sadeghi, Hamed Bagheri, Mohsen Rajaeinejad · Journal of Medical Physics · 2025
Showing 20 of 213 known citations — external sources report more than can currently be individually listed.
Related research
- Assessment of Global Solar Radiation at Selected Points in Nigeria Using Artificial Neural Network Model (ANNM) — shares topic coverage
- Technologies in Texture Analysis – A Review — shares topic coverage
- Evaluation of Plastic Waste Classification Systems — shares topic coverage
- Spectral Analysis of Nonlinear Operators: Theory and Applications to Neural Networks and Optimization — shares topic coverage
- Application of Neural Networks for Predicting the Workability of Self-Compacting Concrete — shares topic coverage
Article metrics
Real usage data collected on this platform.
2
Page views
0
PDF downloads
0
Outbound clicks
213
Citations
Views over time
Views by country
Approximate, from request IP at view time — not citizenship or institution. Countries with fewer than 5 views are grouped as "Other".
Traffic sources
Referring site, by host.
Views and downloads exclude known bots/crawlers. Citations combines this platform's own DOI-resolved index with each external source's own reported total — see Cited by above for individually listed citing works. Last refreshed 0 seconds ago.