Secret Management for AI Agents: Stop Leaking Credentials Into Prompts

If your system prompt contains an API key, you do not have secret management for AI agents. You have a future incident with excellent autocomplete.

Models should decide what needs to happen. Trusted services should decide whether it is allowed, obtain the minimum credential, call the external system, and return a filtered result. That architecture reduces the damage from prompt injection, trace leaks, copied conversations, and plain model mistakes.

Map Every Secret Flow

Document where credentials are created, stored, injected, used, logged, rotated, and revoked. Include development laptops, CI, browser sessions, queues, observability vendors, and support tooling.

Common leaks happen through ordinary convenience:

• Environment dumps attached to bug reports
• Tool errors copied into model context
• Full HTTP headers recorded in traces
• .env files exposed to a filesystem tool
• Long-lived cloud keys injected into containers
• Credentials pasted into chat to “test quickly”

Your threat model should assume untrusted content can influence the model. That includes web pages, emails, documents, issue comments, and tool output.

Keep Secrets Outside the Model Boundary

Give the model a logical tool:

await tools.createInvoice({ customerId, amount, currency });

Do not give it the Stripe key or raw authorization header. The tool service authenticates the agent runtime, evaluates policy, fetches a credential from a secret manager, and performs the request.

The model sees a schema and a filtered result. The executor sees the credential only in process memory for the shortest practical time. Logs see neither.

Prefer Short-Lived Identity

Static API keys are easy to copy and hard to scope. When a provider supports workload identity, OAuth, role assumption, or signed short-lived tokens, use it.

A solid flow looks like this:

1. Agent requests a named capability.
2. Policy checks task, user, target, and risk.
3. Broker exchanges workload identity for a scoped token.
4. Executor performs one bounded action.
5. Token expires quickly and the result is audited.

Do not let the model choose arbitrary OAuth scopes. Define capability bundles in code, such as calendar.read or invoice.refund_under_50, and map them to provider permissions.

Isolate Tools by Trust Level

One giant agent container with database, cloud, email, and payment credentials creates catastrophic blast radius. Split executors by domain. A content-reading worker should not be able to deploy production. A deployment worker should not be able to read customer support attachments.

Use network policy as well as application policy. If a tool only calls one API, restrict egress to that service. If a credential is valid only from a particular workload identity or IP range, enforce it.

Redact Before Data Reaches Observability

Redaction after ingestion is too late. Filter at the point where logs and traces are created.

const SECRET_KEYS = /authorization|api[-_]?key|token|password|cookie/i;

function sanitize(value: unknown): unknown {
  if (Array.isArray(value)) return value.map(sanitize);
  if (value && typeof value === "object") {
    return Object.fromEntries(
      Object.entries(value).map(([k, v]) => [k, SECRET_KEYS.test(k) ? "[REDACTED]" : sanitize(v)])
    );
  }
  return value;
}

Key-name filters are only a first layer. Tokens can appear in URLs, free text, command output, stack traces, and base64 blobs. Add provider-specific patterns, entropy checks, allowlisted fields, and tests with seeded fake secrets.

Handle Filesystem and Subprocess Tools

An agent with shell access can expose secrets without directly opening .env. Commands such as process listings, environment printers, package-manager diagnostics, and crash dumps may reveal them.

Run subprocesses with an explicit environment allowlist. Mount only required files. Keep secret directories outside agent-readable roots. Resolve symlinks before access checks. Return summarized errors instead of raw process state.

Rotate and Revoke Without Drama

Inventory owners, scope, last use, expiry, and rotation procedure for every credential. Automate rotation where providers support overlapping keys. Test revocation: a credential that nobody can safely disable is operational debt.

When a leak is suspected:

1. Revoke or rotate immediately.
2. Search logs and traces using a fingerprint, not the raw secret.
3. Identify uses by workload, time, and destination.
4. Preserve incident evidence without copying the credential again.
5. Fix the exposure path and add a regression test.

Test With Canary Secrets

Seed non-production credentials designed to trigger an alert if they leave the expected boundary. Run prompt-injection tests that request environment variables, headers, or tool configuration. Verify the model refuses, the tool policy blocks access, and monitoring detects the attempt.

Do not grade success only by the model’s response. Inspect traces and executor logs to confirm the fake secret never entered model context.

The Takeaway

Secret management for AI agents is mostly good distributed-systems security applied to an unpredictable planner. Keep credentials outside prompts, exchange workload identity for short-lived capabilities, isolate executors, redact before logging, and make revocation routine.

The model can know that a tool exists. It should not know the key that makes the tool powerful.