Reality Gap Analysis in Physics-Informed ICS Anomaly Detection: From Synthetic Validation to HAI 23.05 Real-World Testbed Evaluation
Physics-informed anomaly detection for industrial control systems (ICSs) is usually validated on synthetic data. How much of that performance survives on real hardware-in-the-loop (HIL) data is an open question. We re-implement PhySec-Edge, a hybrid BiLSTM/β-VAE/GAT/Kalman pipeline, as a full architecture and evaluate it on HAI 23.05. The gap is consistent across five seeds. AI-only F1 falls from 0.774 (synthetic) to 0.525 (HAI); the full hybrid falls from 0.798 to 0.512. The components diverge. PVE residual sharing transfers but weakens on real data (ΔF1 = +0.010 ± 0.008 across five seeds against +0.017 synthetic; ΔMCC = +0.013, positive in every seed). The PVE NIS decision score fails outright (raw FPR = 0.482; a correction brings it to 0.013, but detection does not recover). Under the retrospective fixed-FPR comparison, the preferred configuration is GNN+BiLSTM (F1 = 0.530 ± 0.002), outperforming full-EADE fusion (0.512 ± 0.003). From this split we derive the information-vs-decision pathway distinction, a way to reason about component-level generalization in hybrid architectures. We then set out what it means for physics-informed ICS detector design and edge deployment.