WIP: From Detection to Explanation: Using LLMs for Adversarial Scenario Analysis in Vehicles
David Fernandez
Key Contributions & Takeaways
- Proposes a framework that, instead of just flagging that a driving scene is anomalous, uses an LLM to explain why. Each scene is encoded in a compact domain-specific language and reasoned over against the MUTCD traffic-rule book with zero-shot chain-of-thought, using no labeled training data.
- Introduces AutoSec-X, a dataset of 40 MUTCD-grounded driving scenarios, balanced 20 anomalous / 20 benign, each paired with an expert explanation that names the regulation at stake, so a model is graded on its reasoning, not just its yes/no verdict.
- Detection saturates; explanation separates. Across 9 LLMs, several match on the binary call (87.5%+), but only the Gemini family pairs top detection (92.5%) with grounded explanations that cite the exact MUTCD section. Getting the answer right and explaining why turn out to be two different skills.
The deep networks in a self-driving car can flag that a scene looks wrong, but not why, and they cannot tell a harmless sensor glitch from a deliberate roadside attack.
Encode each scene in a compact domain-specific language and let an LLM reason step by step, checking it against the MUTCD traffic rule book. No labeled training data.
Gemini spots inconsistencies with 92.5% accuracy and, unlike the open models, explains each call by citing the exact regulation it breaks.
This work-in-progress seeded the direction that matured into From Mirage to CLEAR, which scales MUTCD-grounded LLM reasoning to module-level fault attribution on tens of thousands of real scenes.
The framework was benchmarked across 9 LLMs spanning the Gemini, Llama 3.2, and Qwen 2.5 families over the 40-scenario AutoSec-X set, each queried with a single zero-shot chain-of-thought prompt. Detection is scored as accuracy on the balanced anomalous-vs-benign task; explanations are scored against expert ground-truth answers with ROUGE, BLEU, SBERT cosine, and BERTScore. The diagrams above rebuild the paper’s framework and results; exact tables are in the PDF.
“Nine LLMs can all spot a rogue road sign. Only Gemini can tell you which regulation it breaks. Detecting the problem is the easy half; explaining it is where the models split.”
David Fernandez is a PhD candidate in Computer Science at Clemson University, working on safe, efficient, and explainable AI for safety-critical systems. His research spans perception, adversarial robustness, and on-device deployment of large foundation models, including LLMs and VLMs, with five first-authored publications on component-level explainability, zero-shot reasoning, and adversarial scenario analysis, alongside collaborative work on edge AI for industrial agentic systems. Much of this research is grounded in autonomous driving, where trustworthiness, latency, and robustness constraints are unforgiving, but the underlying methods transfer broadly to other high-stakes domains.
As a member of Clemson’s VIPR-GS Research Program, he develops hierarchical LLM reasoning frameworks and VLM evaluation systems for the U.S. Army’s Next Generation Combat Vehicle (NGCV) program, focusing on zero-shot reasoning and component-level explainability under real-world deployment constraints.
At BMW Group, he designs agentic AI systems for enterprise environments, building autonomous prompt optimization pipelines that enable continual agent improvement without model retraining and context-aware moderation frameworks that detect coordinated multi-turn adversarial attacks in production deployments.