An improved hyperparameter optimization framework for AutoML systems using evolutionary algorithms

Abstract Machine learning strategy has changed the face of automated models by integrating themselves into many application domains. The spectrum of applications ranges over various domains, from atmospheric analysis to medical diagnosis. All these applications are design-sensitive, implying that the model's performance depends highly on the selected machine learning algorithm, training procedures, regularization methods, and most importantly, how the hyperparameters are tuned. With the advent of AutoML systems, all these processes including hyperparameter optimization can be automated, producing better performance and faster results. The ideal hyperparameter setting for machine learning models has a direct and significant impact on the model's performance. This paper studies different AutoML models and the hyperparameter techniques used by them for image classification problems. We also discuss some of the libraries for hyperparameter optimization and analyse how they work for different image classification problems. Moreover, the paper proposes a framework for hyperparameter optimization that combines Bayesian optimization and evolutionary algorithms. Experiments are carried out on image classification benchmark datasets to assess the performance of different optimization approaches including the proposed model.