01
Construct representative tool-using agent workflows and define expected behavioral boundaries.
Research projects
Ongoing Research Project
Information Security & Artificial Intelligence
Adversarial Resilience in Agentic AI Systems
An applied security study examining how tool-using AI agents behave when instructions, retrieved content, and connected services contain adversarial input.
Project File
- Lead
- Prof. João Góes
- Period
- 2026 — ongoing
- Status
- Ongoing
- Colleges
- College of Information Security · College of Artificial Intelligence
Research Question
The question organizing the work.
How can agentic systems preserve intended behavior when the information they consume is itself part of the attack surface?
Context
Why this problem is worth examining.
Agentic AI systems increasingly operate across tools, files, APIs, and external information sources. That expands what they can accomplish—but it also expands the number of places where untrusted input can influence decisions.
This project treats the agent not as a chatbot in isolation, but as a small software system with privileges, dependencies, memory, and decision boundaries that can be tested under hostile conditions.
Methodology
How the question becomes testable work.
The methodology is designed to leave behind evidence and reusable artifacts—not only a conclusion.
02
Introduce controlled prompt-injection, retrieval poisoning, permission confusion, and tool-misuse scenarios.
03
Measure whether proposed safeguards reduce unsafe actions without making the agent unusable for legitimate tasks.
04
Document repeatable evaluation cases that can be reused in classroom and engineering environments.
Research Outputs
What the project is designed to leave behind.
Adversarial test harness
Evaluation matrix
Technical report
Teaching cases
Next Steps
Where the investigation goes next.
01
Expand the benchmark to multi-agent workflows and persistent memory.
02
Compare mitigation strategies across different tool and retrieval architectures.
03
Publish a reproducible subset of the evaluation framework for educational use.
Cross-College Research
The problem determines the boundaries—not the org chart.
This project sits across College of Information Security and College of Artificial Intelligence. That cross-college structure is intentional: emerging technology problems frequently combine technical, organizational, legal, and human dimensions.
Keywords
Agentic AIPrompt InjectionRed TeamingAI Safety
Research Projects