AI agent security: a practical guide to securing LLM apps and agents

AI agent security starts from one fact: the model is not a security boundary. Anything an LLM reads can steer it, and anything it writes must be treated as untrusted input. Securing an LLM app or agent therefore means controlling what the surrounding code allows: provider keys that never leave the server, tools scoped to the user and gated by human approval, untrusted content that cannot trigger side effects, validated and encoded output, per-user spending limits, and logs that hold metadata rather than prompts. A system prompt that says “never do X” is a request, not a control.

TL;DR

• Eight failures cover most real LLM security incidents: exposed keys, direct and indirect prompt injection, over-broad tools, PII sent to providers, unescaped output, unbounded spend, prompts in logs and leaked system prompts. Each maps to an OWASP LLM Top 10 2025 entry.
• Design for a fully hijacked model. If injection succeeds, the worst outcome should be a wrong answer, not a deleted database or an exfiltrated inbox.
• Identity, tenant and permission checks belong in code, injected from the session, never chosen by the model.
• Some controls hold better at a gateway: scoped virtual keys instead of provider keys, per-key tool allow-lists, budgets and content-free logs.
• The open-source secure-llm-apps agent skill audits a codebase for all eight with a grep list and an OWASP-mapped checklist: npx skills add build-with-ai-hq/secure-llm-apps.

This is an implementation guide. For the attack class in depth, read The Frontier Wire’s prompt injection explainer; for the full risk list, the OWASP LLM Top 10 guide; for coding agents specifically, Claude Code security.

AI agent security risks at a glance

LLM security and AI agent security overlap: every agent is an LLM app, and agents add tools, loops and autonomy. The table lists the eight failures this guide covers, the OWASP Top 10 for LLM Applications 2025 entry each falls under, and the first fix.

The OWASP GenAI Security Project’s list page still uses the 2025 numbering. The Frontier Wire’s OWASP guide covers the 2026 edition, which renumbers several entries.

API keys in frontends, mobile apps and git

A provider API key in a browser bundle, mobile binary, desktop app or git history is public. Obfuscation does not help. Call the model only from a backend that authenticates users with normal sessions, keep keys in a secret manager, and if a key leaks, revoke it at the provider first; rotation and history purges come after.

Frameworks make this mistake easy. Vite documents that variables prefixed with VITE_ are exposed in client-side source code and should not contain API keys; Next.js inlines NEXT_PUBLIC_ values into the browser bundle at build time. OpenAI’s Node SDK disables browsers by default “to avoid exposing your secret API credentials” and requires an explicit dangerouslyAllowBrowser: true to override.

The secure-llm-apps skill turns this into a search. Its quick diagnosis greps for key patterns (sk-, sk-ant-, AIza), for LLM keys behind NEXT_PUBLIC_, VITE_, EXPO_PUBLIC_ and REACT_APP_ prefixes, and for dangerouslyAllowBrowser, then runs Gitleaks over history. Its verification step greps the production build output (dist/, .next/static/, build/) for key prefixes, which catches keys that only appear after bundling.

Its leaked-key runbook has six steps in a deliberate order: revoke at the provider console; issue a new key into the secret manager and redeploy; review provider usage for the leak window; purge git history with git filter-repo; add pre-commit and CI secret scanning with Gitleaks or TruffleHog; and find the path that leaked it. The skill puts it plainly: “Purging history without revoking does nothing.” OpenAI’s safety best practices give the same first step: revoke promptly and replace.

Direct and indirect prompt injection

Direct injection comes from the user typing into the app. Indirect injection arrives in content the model reads on the user’s behalf: a RAG chunk, PDF, email, calendar invite, GitHub issue, web page or tool result. No classifier reliably removes it, so the design goal is that a hijacked model still cannot cause damage.

OWASP’s LLM01 entry defines indirect injection as occurring “when an LLM accepts input from external sources, such as websites or files,” and notes that injected content need not be visible to humans. The skill’s prompt-injection reference adds a third kind, stored injection: a ticket or shared document with hidden instructions that an agent later reads while acting for an administrator. Its threat model names the hiding places: white text, HTML comments, alt text, zero-width characters, base64 and other languages.

The reference ranks defenses by what actually reduces risk: capability limits first, then authorization in code, human approval for side effects after untrusted content enters, egress control, output validation, privilege separation, prompt hygiene, and detection last. Two patterns carry most of the weight.

• Taint tracking. The session is marked tainted the moment a tool such as web_fetch, read_email or search_docs returns content. From then on, write and destructive tools need approval and fetch tools are restricted. The skill gives Python and TypeScript versions with a 15-step cap.
• Dual-model quarantine. For flows that must read untrusted content and act, a quarantined model with no tools reads the content and returns only a narrow schema (an enum category, an ID matching a pattern, a bounded number). A privileged model or plain code acts on those fields and never sees the raw text. The reference warns that free-text fields passed between them “re-open the channel.”

Delimiting untrusted content in <untrusted> tags and telling the model not to follow instructions inside them is in the skill too, labelled “useful, never sufficient.” Simon Willison’s lethal trifecta explains why containment matters more than filtering: an agent with private data, untrusted content and a way to communicate externally can be made to leak. The prompt injection explainer covers real CVEs and testing.

Over-broad and destructive tools

An agent should hold the smallest tool set its job needs, with identity and tenant arguments injected from the session rather than chosen by the model, and every destructive action (delete, external email, payment, deployment, shell) confirmed by a human who sees the exact arguments. Loops need step, tool-call and time limits.

OWASP’s Excessive Agency entry traces the root cause to excessive functionality, excessive permissions or excessive autonomy, and its prevention list includes human-in-the-loop approval and “complete mediation”: authorization enforced in downstream systems rather than decided by the model. The skill’s dispatcher implements this in about fifteen lines. Each tool carries a risk class of read, write or destructive; the dispatcher overwrites tenant_id from the session, checks ctx.user.can(name, args), and requests approval for destructive tools, or for write tools once the session has seen untrusted content:

def dispatch(name: str, args: dict, ctx) -> dict:
    tool = TOOLS.get(name)
    if tool is None:
        return {"error": f"unknown tool {name}"}
    args = {**args, "tenant_id": ctx.tenant_id}           # scope from session, not model
    if not ctx.user.can(name, args):
        return {"error": "forbidden"}
    if tool.risk == "destructive" or (tool.risk == "write" and ctx.saw_untrusted_content):
        if not ctx.request_approval(name, args):           # UI shows exact args
            return {"error": "user declined"}
    return tool.fn(**args)

Two of the skill’s common mistakes belong here: letting the model choose user_id, tenant_id, file paths or table names, and approval prompts that show “a summary written by the model instead of the actual arguments.” The second matters because an injected model can write a reassuring summary of a harmful call.

MCP servers make over-broad tools the default, since many expose write tools next to read tools. The MCP specification says clients should prompt for confirmation on sensitive operations and show tool inputs before calling. The sibling MCP best practices guide covers per-agent tool budgets and allow-lists in detail.

PII and secrets sent to providers

Every prompt is a data transfer to a third party. Send only the fields a task needs, redact or pseudonymize personal data locally before the call, scan for pasted credentials, and confirm the provider account’s retention and training settings. Retrieval must be filtered by tenant and permissions inside the query, not after the model sees results.

OWASP’s LLM02 entry lists the data at stake: personal information, financial details, health records, confidential business data, security credentials and legal documents. The skill’s pii-and-data reference opens with a decision table: which fields the task needs, whether the data is regulated (then contract terms, region and retention come first), whether the answer must reference the entity (reversible pseudonymization) or not (irreversible redaction), and who can read stored prompts.

For redaction it uses Presidio, the open-source PII framework, which runs on the team’s own infrastructure so detection happens before data leaves. The reference provides Python code for redaction with a custom recognizer, a reversible pseudonymizer that swaps entities for tokens such as <PERSON_1> and restores them after the call, and a TypeScript path through Presidio’s REST containers with a fail-closed rule: “if the redaction service is down, do not send the raw text.” It is candid about limits: Presidio “will miss some PII,” so it pairs with field minimization.

Secrets need their own scan because, as the reference notes, users paste API keys, .env files, connection strings and private keys into chats constantly. It ships regex patterns for OpenAI, Anthropic, AWS, GitHub, Google and Slack tokens, private keys, connection strings and JWTs, masks matches, and tells the user to rotate. Multi-tenant rules close the section: one index per tenant or a mandatory metadata filter in code, and conversation memory and semantic caches keyed by tenant so one user’s content never answers another’s prompt.

Unescaped LLM output in SQL, HTML and shell

LLM output is user input. Parse it into a strict schema, fail closed when it does not validate, then encode it for its destination: parameterized SQL, escaped HTML, sanitized markdown with allowlisted link and image hosts, argv-list shell commands, and sandboxed code execution. Never pass model text to eval, innerHTML or shell=True.

OWASP’s LLM05 entry lists both classic cases, JavaScript or markdown interpreted by a browser as XSS and “LLM-generated SQL queries are executed without proper parameterization,” and recommends encoding output and strict Content Security Policies. The skill gives Pydantic and Zod schemas with enums, regex patterns and numeric bounds, and a sink table:

Markdown deserves special attention because it is an exfiltration channel. A rendered markdown image whose URL points at an attacker host, with stolen data in the query string, sends that data when the page loads, with no click. The skill’s prompt-injection reference includes DOMPurify and nh3 hooks that strip src and href attributes pointing outside an allowlist. Its verification step feeds <img src=x onerror=alert(1)> and '; DROP TABLE users;-- through every render and query path as if the model had produced them.

Unbounded spend and abuse

Unbounded consumption is both a cost risk and a denial-of-service vector. Authenticate every LLM endpoint, rate limit per user and per tenant on requests and tokens, add a global ceiling, cap input size, max_tokens, agent steps, retrieved chunks and timeouts, and set hard budgets with alerts on spend velocity.

OWASP’s LLM10 entry names “Denial of Wallet,” where attackers exploit the cost-per-use model of hosted AI services, and recommends rate limiting and user quotas. OpenAI’s safety guidance makes a related point: limiting output tokens “helps reduce the chance of misuse,” and safety identifiers, hashed per-user IDs sent with requests, help trace abuse to one user.

The skill supplies a fixed-window Redis limiter in Python and TypeScript, and lists the mistake that defeats most limiters: “Rate limiting by IP only (trivially bypassed) instead of by authenticated user and tenant.” Its verification script sends twice the per-user limit and confirms a 429 for that user while others are unaffected. The rate limiting guide covers algorithms and LLM token limits, and the sibling LLM cost optimization guide covers the cost side.

Prompts and completions in logs

Logs are a quiet data leak: they copy prompts, completions and sometimes Authorization headers into systems with wider access and longer retention than the app. Log metadata by default, and keep content only when redacted, access-restricted and short-lived. Check tracing, APM and error trackers, which often capture full request bodies.

The skill’s default log line holds a request ID, a SHA-256 hash of the user ID, the model, input and output token counts, tool names and outcome, and nothing the user typed. Its audit greps look for logger calls containing prompt, messages, completion or choices, for any logging of Authorization or x-api-key, and for tracing integrations (sentry, datadog, opentelemetry) whose capture settings need checking. Two of its common mistakes come from this area: redacting PII in the prompt while logging the original request body upstream, and error messages that echo prompts or provider errors back to users.

System prompts belong in the same discussion. OWASP’s LLM07 entry states that “the system prompt should not be considered a secret, nor should it be used as a security control.” The skill assumes extraction: no secrets, internal URLs or authorization rules in the prompt, plus a random canary string per deployment that triggers an alert if it ever appears in output.

Controls best enforced at a gateway

Some controls are stronger at a gateway than in each app, because they still hold when an app is compromised or an agent is prompt-injected. The skill uses Bifrost, an open-source (Apache 2.0) AI gateway, for four of them: scoped keys, tool allow-lists, budgets and rate limits, and content-free logs.

The pattern is that apps never hold a provider key. Provider keys live only in Bifrost, and each service, team or developer gets a virtual key with an allowed_models list, optional expiry, and its own budget and rate limit on requests and tokens. App code changes only its base URL and key. The skill is careful to add that “a virtual key is still a bearer credential,” so it stays server-side too; its value is that it is scoped, capped, expirable and revocable without touching provider accounts.

For agents, each key can carry a per-key MCP tool allow-list. Bifrost’s documentation states that a key without MCP configuration gets no MCP tools, that a request header can only narrow the list, never widen it, and that the list is enforced again at tool execution time, so an injected agent cannot request its way to more tools. For logs, the client configuration setting disable_content_logging keeps usage metadata and drops content, and the skill notes that each observability connector has its own flag that must be set as well.

A gateway holds every provider key, so the skill’s keys-and-gateway reference treats it like a secrets store: admin auth on with a setup token for the first admin, enforce_auth_on_inference on, direct keys off, explicit CORS origins, a private network with TLS, and a pinned version. Enterprise tier. Managed PII, secrets and content-safety guardrails can also be applied centrally at the gateway. Other approaches to guardrails are compared in the AI guardrails tools guide.

The secure-llm-apps skill

secure-llm-apps is an MIT-licensed Agent Skill that audits and hardens LLM apps and agents against the practical OWASP LLM Top 10 risks. Claude Code, Codex, Cursor and other compatible agents load it when a request matches its description, then run its diagnosis, report findings and apply fixes.

It is published by the build-with-ai-hq GitHub organization, created in October 2026, at github.com/build-with-ai-hq/secure-llm-apps, and listed on skills.sh. Its contents:

• SKILL.md (2,695 words): a six-part quick diagnosis, four triage questions (where the credential lives, what untrusted text reaches the prompt, what the model can cause, what limits exist), seven core practices with Python and TypeScript fixes, the gateway section, a seven-point verification list and eight common mistakes.
• references/audit-checklist.md: the full grep list, defined once as a g() helper and grouped into keys and secrets, prompt construction and untrusted content, tools and agents, output sinks, consumption, data and logging, and gateway or MCP deployment; a checklist mapped to LLM01 through LLM10 plus keys, infrastructure and logging; and a report template with severity, location, category, risk, exploit sketch, fix and verification step.
• references/prompt-injection.md: threat model, ranked defenses, taint tracking, the quarantine pattern, exfiltration channels, canaries, MCP-specific risks and test payloads.
• references/keys-and-gateway.md and references/pii-and-data.md: the leaked-key runbook, gateway lockdown, Presidio redaction, secret patterns, multi-tenant retrieval and logging hygiene.

A sample of the quick diagnosis, run from a repository root:

grep -rnE "(NEXT_PUBLIC|VITE|EXPO_PUBLIC|REACT_APP)_[A-Z_]*(OPENAI|ANTHROPIC|GEMINI|GOOGLE|MISTRAL|GROQ|COHERE|LLM|AI)[A-Z_]*KEY" .
grep -rn "dangerouslyAllowBrowser" --include=*.{ts,tsx,js,jsx} .
grep -rnE "dangerouslySetInnerHTML|innerHTML\s*=|v-html|\|\s*safe\b|mark_safe\(" .
grep -rnE "max_tokens|max_output_tokens|maxTokens|max_iterations|maxSteps|recursion_limit" .

The skill treats each hit as “a lead, not a verdict” and reads the surrounding code. Install it with the skills CLI, covered in the guide to npx skills and skills.sh:

# Project install, with interactive prompts
npx skills add build-with-ai-hq/secure-llm-apps

# Target specific agents and skip prompts
npx skills add build-with-ai-hq/secure-llm-apps -a claude-code -a codex -a cursor -y

# Global (user-level) install, or preview without installing
npx skills add build-with-ai-hq/secure-llm-apps -g
npx skills add build-with-ai-hq/secure-llm-apps --list

# List installed skills; update later
npx skills list
npx skills update

In a test install on October 28, 2026, The Frontier Wire ran npx -y skills@1.7.4 add build-with-ai-hq/secure-llm-apps -a claude-code -a codex -a cursor -y in an empty git repository on macOS. The CLI reported “Installed 1 skill,” placed the files in .agents/skills/secure-llm-apps for Codex and Cursor, symlinked them for Claude Code, recorded the source and a computed hash in skills-lock.json, and printed: “Review skills before use; they run with full agent permissions.” The lint_skills.py linter from the sibling agent-skills-for-teams skill reported no errors or warnings for it. The test covered installation and file inspection; the skill was not run against a production application.

Using the skill in practice

The skill loads when a request matches its description, which lists phrases such as “is my AI app secure”, “security review of our chatbot/agent/RAG”, “my OpenAI key leaked”, “can I call the LLM from the browser”, “the agent deleted something”, “redact PII before sending to OpenAI/Anthropic”, “someone is burning our API credits” and “OWASP LLM Top 10”. Prompts that match:

• “Run a security review of our RAG chatbot against the OWASP LLM Top 10 and write up findings with the report template.”
• “My OpenAI key leaked in a commit. Walk me through the fix and make sure it cannot happen again.”
• “Redact PII before we send support tickets to Anthropic, and stop logging prompts.”
• “Someone is burning our API credits. Add per-user rate limits and budgets.”

For teams whose agents use MCP heavily, the sibling mcp-gateway-best-practices skill goes deeper on tool budgets, transports and approval, as the MCP best practices guide describes; for coding agents, ai-coding-agent-governance covers per-developer keys and rollout.

Scope of the skill and this guide

The skill works as a security reviewer inside the agent: it audits code, applies fixes and sets up gateway controls, and it pairs with runtime guardrails and scanners rather than standing in for them. Like any skill, it runs with the agent’s permissions, so the CLI’s advice to review skills before use applies. Its gateway examples use Bifrost; the vendor-neutral fixes apply on any stack.

Its depth follows where most incidents start. Keys, injection, tools, output handling, spend and logs get full treatment, while data poisoning, vector store isolation and misinformation are covered as checklist items. It complements a penetration test, product-specific threat modelling and provider-side settings such as retention agreements, and its checklist reminds teams to confirm those. The OWASP mapping here uses the 2025 numbering on genai.owasp.org; teams that report against the 2026 edition should translate.

The pattern behind every section of this guide is the same: assume the model will be fooled, then make sure being fooled is cheap. Server-side keys, session-scoped tools, approval after untrusted input, validated output, per-user limits and metadata-only logs are each ordinary engineering, and together they are what AI agent security means in practice. The secure-llm-apps skill packages that checklist in a form an agent can run against a codebase; npx skills add build-with-ai-hq/secure-llm-apps installs it, and its files are worth reading first.