Sparse kernel prototype-based classifiers: new contributions

Abstract

The main purpose of this thesis is to contribute to the development of prototype-based classifiers using kernel methods. Initially, the application of the kernel trick to the artificial neural network self-organizing map (SOM) is investigated for the classification of batch datasets (in which all training data are simultaneously available), also verifying the influence of different kernel functions and labeling methods. The results of these initial experiments showed that using kernel functions other than the commonly employed ones (Gaussian and Linear) can lead to superior performance. Furthermore, it was verified that, in the analyzed datasets, there are advantages in computing both the distance measure and the definition of the winning prototype in the feature space. Then, considering that defining the number of prototypes is very important for the accuracy of prototype-based classifiers, this work proposes the use of sparsification methods, such as approximate linear dependence (ALD), novelty, surprise and coherence, for the automatic selection of this quantity, employing the weighted K-nearest neighbors algorithm as the classifier. These methods are compared with respect to their influence on classifier accuracy and on the number of prototypes selected. In the experiments carried out with the proposed model, here referred to as sparse kernel (SPARK), no sparsification method or kernel function consistently outperformed the others across all datasets. These findings highlight that the effectiveness of the model depends on the dataset characteristics, reinforcing the need for a careful selection of sparsification methods, kernel functions, and hyperparameters adapted to each specific task. Moreover, in most cases, the performance was superior to that of the kernel-SOM algorithm, in which the definition of the number of prototypes implies a higher computational cost. Finally, in many recent applications, data may be provided as a continuous stream. In this context, a new theoretical framework, here referred to as sparse online kernel (SPOK), is proposed for building adaptive prototypebased models using kernel methods. The proposed model was evaluated in the classification of synthetic and real data streams. The results showed that, even relying solely on prototypes and K-nearest neighbor strategies, the model achieved, with only a few prototypes, performance comparable to the state-of-the-art in data stream classifiers, which are often based on data windowing, ensembles or decision trees.