Enhancing Accuracy with Recursive Feature Selection Using Multiple Machine Learning and Deep Learning Techniques on NSL-KDD Dataset

The world has moved toward digital revolution and more and more services are now being available online. This has presented significant challenges in ensuring the availability, integrity, and confidentiality of networks. Therefore, the implementation of intrusion detection systems (IDS) that actively monitor network activity and dynamically analyze incoming traffic has become essential. In modern computer networks, IDS play a critical role in identifying and alerting administrators of any malicious activities that could threaten data security.This research aims to evaluate machine learning (ML) and deep learning (DL) techniques for intrusion detection using the NSL-KDD dataset. While having many features may seem advantageous, it doesn’t always lead to improved performance on large datasets. Reducing and picking just the right set of features may improve both speed and accuracy. This is why a method called recursive feature elimination (RFE) is utilized to select the most relevant features for the NSL-KDD dataset. In order to test the efficacy of ML and DL methods in IDS, we have undertaken a comprehensive experiment. Specifically, we tested the effectiveness of several algorithms, including CNN, RNN, LSTM, GB, CART, and CatBoost. Our study involved evaluating the performance of various algorithms and comparing them, and the detection accuracy [lowest-highest] for all the attacks are as follows: DoS [98.79–99.81%], Probe [96.34–99.64%], R2L [96.15–98.66%], U2R [79.98–99.73%].