Banana Navy
← Catalog FR
Lab · AI threat modeling · Detailed card

Improper output handling

reusing the model answer downstream without validating it, up to critical components.

SheetF22
CategoryAI1-TB7 → AI6-TB7 (output driving an action)
Layers14 · LLM Interpretation, 15 · Policy Engine, 16 · Workflow Execution
SystemAI voicebot

treat the model output as untrusted input.

The threat

the LLM response is reused downstream without validation: HTML/JS rendering (XSS), query building (SQLi), string handed to a shell or interpreter (code execution), tool or API input (tool command injection), constructed URL (SSRF). Since the output is attacker-influenceable (via injection, see F15/F16), it becomes the impact vector: the real damage happens at the downstream component.

Blind spotWhy classic frameworks miss it

classic grids validate the prompt input, never the output; they treat model answers as trusted data flowing to components. An unsanitized output turns an injection into real impact (XSS, SQLi, RCE, tool abuse) as soon as it reaches a critical component.

MitigationProposed approach

treat model output as untrusted input. Per-sink validation and sanitization: HTML/JS encoding, parameterized queries, typed schemas and allowlists for tool inputs, no direct execution of the answer, context isolation (CSP), logging of output to sink flows.

The proposed control
no model output reaches a critical component without passing a deterministic validator.

Expected evidence
a tampered output (via injection) produces no harmful action or sink; end to end test from adversarial input to output to component.

SourcesReferences and public research

MITRE ATLAS 2026.07AML.T0051 LLM Prompt Injection ✓ (chained) · AML.T0053 AI Agent Tool Invocation ✓ (output driving a tool)
OWASP GenAIOWASP GenAI LLM05:2025 Improper Output Handling
Public researchGreshake et al., "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection" (arXiv 2302.12173) ; "From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?" (arXiv 2308.01990) : an influenceable output reaches a component when no output guard exists.
Related cardsF15 · Direct injection · F16 · Indirect injection · F20 · Tool abuse
layer 14 · LLM Interpretationlayer 15 · Policy Enginelayer 16 · Workflow Execution
Open the mapThe 20 checkpoint map
AI Threat Modeling Catalog

Explore the 20 security layers

The 20 checkpoint map

MITRE ATLAS 2026.07 · OWASP GenAI · risk-voicebot