A comparative analysis of undersampling techniques for network intrusion detection systems design

Abstract

Intrusion Detection Systems (IDS) figure as one of the leading solutions adopted in the network security area to prevent intrusions and ensure data and services security. However, this issue requires IDS to be assertive and efficient processing time. Undersampling techniques allow classifiers to be evaluated from smaller subsets in a representative manner, aiming high assertive metrics in less processing time. There are several solutions in literature for IDS projects, but some criteria are not respected, such as the adoption of a replicable methodology. In this work, we selected three undersampling methodologies: random, Cluster centroids, and NearMiss in two novel unbalanced datasets (CIC2017 and CIC2018) for comparison between five classifiers using cross-validation and Wilcoxon statistical test. Our main contribution is a systematic and replicable methodology for using subsampling techniques to balance the data sets adopted in the IDS project. We choose three metrics for classifier's choice in an IDS design: accuracy, f1-measure, and processing time. The results indicate that the under-sampling by Cluster centroids presents the best performance when applied to distance-based classifiers. Moreover, under-sampling techniques influence the process of choosing the best classifier in the design of an IDS.