On local models for novelty detection: new algorithms and practical applications

Abstract

Machine Learning (ML) has redefined problem-solving by enabling systems to autonomously learn and make decisions from data patterns. With the increasing complexity of data, there is a growing need for sophisticated ML techniques. In this regard, local learning has emerged as a promising approach that concentrates analysis on specific localized characteristics of the data, such as data proximity, feature subsets, or graph structures. Unlike global learning, which aims to build a single global model, local learning concentrates on smaller, potentially more interpretable problem subsets. This thesis explores the effectiveness, advantages, and limitations of this approach across different scenarios. Additionally, it introduces two novel local learning techniques designed specifically for one-class classification: local kernel principal component analysis (LKPCA) and local autoencoder (LAE). LKPCA leverages kernel methods to handle nonlinear data, while LAE utilizes deep autoencoders (DAEs). The proposed local learning techniques have the potential to reduce processing costs by leveraging localized representations, which makes them particularly efficient in handling imbalanced datasets and redundant data. Moreover, they are effective at removing noise and irrelevant data in sparse regions, enabling the model to focus on meaningful patterns and improve the detection performance. LKPCA variants were compared against global KPCA and state-of-the-art methods across 17 one-class datasets derived from 9 benchmark datasets. The results indicate that cooperative LKPCA generally outperforms global KPCA, while competitive LKPCA frequently presents lower performance. The cooperative LKPCA also demonstrated higher predictive power compared to state-of-the-art methods on several datasets. Regarding LAE, it was assessed on 7 time series datasets, revealing an improved F1-score over global autoencoders (AEs) in datasets such as BeetleFly, Wafer, and ItalyPowerDemand. These results show the potential of applying local learning structures in one-class classification problems.