Hyperparameter selection in support vector machines

OData support
Dr. Horváth Gábor
Department of Measurement and Information Systems

The Support Vector Machines an efficient supervised learning algorithm, that applicable to linear and nonlinear classification and regression problems.

The SVM provides sparse solution, and maximalize the geometric margin.

The efficiency of SVM depends on some parameters used in the learning process.

The focus of this paper is to investigate a opportunities of choosing optimal hiperparamters.

First in this paper I consider the grid search algorithm as the exhaustive search of the parameter space, while mapping the estimated error provided by each combination of the parameters.

This method enables to understand the effects of the parameters to the SVM solution, and present a promising fact that one does not need to find the optimal parameters, because there are a wide area in the parameter space that gives similar estimated error rate, then the optimal solution.

The second part presents gradient based methods for tuning hyperparameters.

The key concept is to determint the gradient of the validation function of the SVM, which enables to use gradient base optimization in the parameter space, while minimizing the estimated error.

The last part investigates the possibilities of using the radius-margin bound for iterative methods of parameter selection.


Please sign in to download the files of this thesis.