Particle Swarm Optimization Setup for Hyperparameter Selection in Support Vector Machines

Abstract Selecting hyperparameters for support vector machines is an optimization task that affects their classification performance. In particular, particle swarm optimization has been successfully applied for this purpose; however, the literature in the field has not addressed the effect of the population topology and the method for handling constraints in the search space. In this paper, we assessed two population types (von Neumann and fully connected) and four methods for handling constraints in the search space (penalization, dynamic penalization, reflect, and border), for a total of eight different PSO setups. These PSO setups helped us to optimize an SVM using a radial basis function kernel, which means that two hyperparameters were selected: cost parameter (C) and kernel width (γ). We conducted the experiments over 10 benchmark classification problems and for atrial fibrillation detection. Our findings suggest that using a von Neumann population together with the reflect method provides the SVM with the best classification capabilities and shortest execution time.