Effect of Dataset Partition and Normalization Methods in the Hyperparameters Optimization Phase on the Efficiency of The Levenberg-Marquardt Algorithm

Multilayer perceptron neural networks are a family of continuous functions and offer a great flexibility for modeling empirical data. The lack of attention to the choice of optimal parame- ters(partition, sample size, number of units in the hidden layer, data normalization method) in the building of neural models negatively influences their predictive and explanatory performance.The present study aims to evaluate the effect of partition and normalization methods on the optimization phase of hyperparameters using the Levenberg Marquardt (LM) algorithm in aprediction context. The Monte Carlo approach was used to train several datasets generatedby varying the internal structure of a 3-MLP from simple to complex with the LM algorithm for different partition rates and different methods of normalizations most commonly used. A total of 995880 models were built and compared on the basis of R2 and MAPE criteria. The results showed that the application of the partitioning rate 85% - 15% for training and testing respectively, the normalization method minmax , a learning rate of 0.25 for the training of the algorithm with nine (9) neurons at the hidden layer with the application of the sigmoid at the hidden layer as well as at output layer led to their best performances.