Data structure better than labels? Unsupervised heuristics for SVM hyperparameter estimation

Abstract

Classification is one of the main areas of pattern recognition research, and within it, support vector machine (SVM) is one of the most popular methods outside of the field of deep learning – and a de facto reference for many machine learning approaches. Its performance is determined by parameter selection, which is usually achieved by a time-consuming grid search cross-validation procedure (GSCV). That method, however, relies on the availability and quality of labelled examples and thus, when those are limited, can be hindered. To address this problem, several unsupervised heuristics exist that utilise the characteristics of the dataset to select parameters, rather than relying on class label information. While being an order of magnitude faster, they are scarcely used under the assumption that their results are significantly worse than those of grid search. To challenge that assumption, we have surveyed several heuristics for SVM parameter selection and tested them against GSCV on over 30 standard classification datasets. The results demonstrate their high accuracy, with performance in terms of statistical significance comparable to GSCV, opening up an avenue for reliable label-free model defaults in resource-constrained settings, e.g., edge devices or rapid prototyping.