Benchmarking AI-Based Intrusion Detection Models for Cyber-Physical Systems: A Dataset-Driven Analysis

Abstract

Cyber-Physical Systems (CPS) are increasingly deployed across sectors such as energy, manufacturing, and healthcare, where real-time monitoring and secure operations are essential. As these systems become targets for sophisticated cyber threats, the need for accurate, low-latency intrusion detection has become critical. While many studies have proposed AI-based solutions for securing CPS, there remains a lack of systematic benchmarking across diverse datasets and attack scenarios.This paper presents a dataset-driven benchmarking study of machine learning and deep learning-based Intrusion Detection Systems (IDS) tailored for CPS environments. Using publicly available CPS-related datasets—including UNSW-NB15, CICIDS2017, BATADAL, and the ICS-Cyber Attack Dataset—we evaluate the performance of Support Vector Machines (SVM), Random Forest (RF), Long Short-Term Memory (LSTM) networks, and a proposed Hybrid AI model. Evaluation metrics include accuracy, precision, recall, F1-score, and false positive rate, providing a holistic view of each model's effectiveness. Results indicate that while traditional models like RF and SVM offer faster inference times, deep learning models such as LSTM consistently outperform in terms of detection accuracy and false positive reduction. The Hybrid AI model demonstrates a balanced trade-off between performance and efficiency, making it a promising approach for real-world CPS deployments. This benchmarking effort serves as a foundation for selecting and optimizing IDS solutions in CPS, highlighting the importance of aligning detection models with dataset characteristics and operational constraints.