Addressing key challenges of adversarial attacks and defenses in the tabular domain: A methodological framework for coherence and consistency

Machine learning models trained on tabular data are vulnerable to adversarial attacks, even in realistic scenarios where attackers only have access to the model's outputs. Since tabular data contains complex dependencies among features, it presents a unique challenge for adversarial samples which must maintain coherence and respect these dependencies to remain indistinguishable from benign data. Moreover, existing attack evaluation metrics—such as the success rate, perturbation magnitude, and query count—fail to account for this challenge. To address these gaps, we propose a technique for perturbing dependent features while preserving sample coherence. In addition, we introduce Class-Specific Anomaly Detection (CSAD), an effective and novel anomaly detection approach, along with concrete metrics for assessing the quality of tabular adversarial attacks. CSAD evaluates adversarial samples relative to their predicted class distribution, rather than a broad benign distribution. This ensures that subtle adversarial perturbations, which may appear coherent in other classes, are correctly identified as anomalies. We extend CSAD for importance-based anomaly detection by integrating SHAP explainability techniques to detect inconsistencies in model decision-making. Our evaluation of adversarial sample quality incorporates both anomaly detection rates and importance-based assessments to provide a more comprehensive measure. We evaluate various attack strategies, examining black-box query-based and transferability-based gradient attacks across four target classification models. Experiments on benchmark tabular datasets reveal key differences in the attacker's risk effort and attack quality, offering insights into the strengths, limitations, and trade-offs faced by attackers and defenders. Our findings lay the groundwork for future research on adversarial attacks and defense development in the tabular domain.