---
title: "AI security, prompt injection, data leakage, and guardrails"
chapter: "21"
---

# AI security, prompt injection, data leakage, and guardrails

Models process instructions and data in the same medium, so untrusted text can
attempt to redirect behavior. Prompt injection is a system-design problem, not
a phrase a system prompt can permanently solve.

## Threat paths

- direct malicious user instruction;
- indirect instruction in retrieved documents, web pages, emails, or tool data;
- data exfiltration through tool arguments or model output;
- unauthorized retrieval across tenants;
- excessive agency or confused-deputy tools;
- poisoned training/embedding data;
- denial of wallet through long prompts or loops;
- insecure model/MCP supply chain.

## Controls

Separate trusted policy from untrusted content. Minimize context and tools.
Authorize tools server-side. Validate input/output schemas. Use egress
allowlists, tenant filters, rate/cost limits, sandboxed execution, approval
gates, audit logs, and secret redaction.

## Guardrails

Input/output classifiers, regex/rules, policy models, schema validation, and
human review are layered controls. A guardrail model can fail and must not be
the only authorization boundary.

## RAG injection

Retrieved content is evidence, never authority. Strip active markup, label
source boundaries, tell the model not to follow embedded instructions, and test
documents that contain realistic injection attempts.

## Data governance

Classify prompts, files, embeddings, traces, tool results, and feedback. Define
provider retention, region, encryption, access, deletion, and incident
response. Avoid logging raw prompts by default.

## Red team

Test prompt extraction, cross-tenant queries, encoded attacks, tool escalation,
malicious MCP servers, long-loop attacks, citation spoofing, and exfiltration.
Turn every confirmed failure into a regression test.

## Feynman check

The model reads both the manager's policy and a stranger's note. The system
must make it impossible for the stranger's note to unlock the cash drawer.