Threat Modeling

Scope: design-time identification and prioritization of threats — data-flow modeling, STRIDE, attack trees, LINDDUN privacy analysis, risk scoring, and integration into delivery.

Why and when

Threat modeling finds design flaws that scanners cannot: broken trust assumptions, missing authorization, and abuse paths. Do it when a system is new, when a trust boundary moves, when a new data class or actor (including an AI agent) appears, and when a significant dependency or deployment model changes. One focused session beats a document nobody reads.

The four questions every method answers:

1. What are we building?
2. What can go wrong?
3. What are we going to do about it?
4. Did we do a good enough job?

Choosing a method

| Method | Best for | Cost | Output | |---|---|---|---| | STRIDE | Fast per-element analysis of a design | Low | Threat list mapped to mitigations | | Attack trees | One high-value asset or goal, depth over breadth | Medium | Structured attack paths and costs | | LINDDUN | Privacy and personal data (GDPR-style review) | Medium | Privacy threats and controls | | PASTA | Risk-centric, business-objective-driven, regulated systems | High | Stage-by-stage risk analysis | | OCTAVE | Organizational risk, not a single system | High | Practice-level risk profile | | Abuse cases | Requirement-stage, agile teams | Low | Negative requirements and tests |

Pick one primary method and one secondary. Most product work runs STRIDE plus LINDDUN; a high-value asset (signing key, payment flow) gets an attack tree.

Data-flow threat modeling

Model four element types and mark every place data crosses an identity, network, or privilege boundary.

| DFD element | Meaning | STRIDE focus | |---|---|---| | External entity | Actor or system outside your control | Spoofing, repudiation | | Process | Code that handles data | All six | | Data store | Persistence (DB, bucket, queue, vector index) | Tampering, information disclosure, repudiation | | Data flow | Data in transit between elements | Tampering, information disclosure, denial of service |

Trust boundaries are the lines between different levels of control: user to edge, edge to service, service to database, service to third party, workload to LLM provider, agent to tool. Threats concentrate on boundaries; controls belong there too.

Practical steps:

1. Draw the current and next-state diagrams at the same level of abstraction; label protocols and data classes on every flow.
2. Mark trust boundaries explicitly, including inside the cluster (namespaces, service mesh, policy domains).
3. Enumerate threats per element with STRIDE; for each, record the abuse path, existing control, gap, and owner.
4. Turn mitigations into requirements and tests; every threat without a control becomes a backlog item or a written exception.
5. Revisit the model at each significant change, not annually.

STRIDE per element

| Threat | Question | Typical controls | |---|---|---| | Spoofing | Can an actor pretend to be another? | Strong authn, mTLS, workload identity, signed tokens | | Tampering | Can data or code be modified in transit or at rest? | Integrity checks, AEAD, signed artifacts, immutable stores | | Repudiation | Can an action be denied? | Audit logs, time-stamped events, non-repudiable signatures | | Information disclosure | Can data leak? | Classification, encryption, least privilege, redaction | | Denial of service | Can availability be degraded? | Rate limits, quotas, timeouts, autoscaling, backpressure | | Elevation of privilege | Can a principal gain rights it should not have? | Central authz, least privilege, sandboxing, no ambient authority |

Element matrix (use as a worksheet):

| Element | S | T | R | I | D | E | |---|---|---|---|---|---|---| | External entity | x | | x | | | | | Process | x | x | x | x | x | x | | Data store | | x | x | x | x | | | Data flow | | x | | x | x | |

For each x, write one sentence: "A malicious X can Y, leading to Z." That is a threat statement; if you cannot write it, the threat does not apply.

Attack trees

Attack trees decompose a goal into alternative and sequential steps. Use them for a single asset: "forge a signing key", "exfiltrate the customer database", "move funds without detection".

• Root: attacker goal, stated as an outcome.
• Children: OR nodes for alternatives, AND nodes for required steps.
• Leaves: atomic actions with a cost, skill level, and detection risk.
• Prune branches that existing controls make infeasible; annotate the cheapest surviving path.
• The cheapest path is the priority, not the longest one.

Example (password reset takeover):

Goal: take over a victim account
OR
  - Reset token theft
    AND - obtain token (referrer leak, log, shared mailbox)
    AND - use before expiry/rotation
  - Reset flow logic bypass
    AND - discover host header / IDOR weakness
    AND - trigger reset for victim
  - SMS OTP interception
    AND - SIM swap or SS7 access
  - Help-desk social engineering
    AND - impersonate victim convincingly

Controls fall out directly: bind tokens to the request, single use, short TTL, ignore Host headers for link generation, prefer passkeys for recovery, log and rate-limit reset, train help desk with challenge questions that are not public data.

LINDDUN and privacy

LINDDUN drives privacy threats from data flows. Run it for anything processing personal data or behavioral telemetry.

| Letter | Threat | Question | Controls | |---|---|---|---| | L | Linking | Can data about a person be linked across contexts? | Pseudonyms, purpose separation, unlinkable identifiers | | I | Identifying | Can a person be identified from data? | Anonymization, k-anonymity/differential privacy, minimize identifiers | | N | Non-repudiation | Can a person be forced to own an action? | Deniable schemes where appropriate; careful audit scope | | D | Detectability | Can presence or behavior be inferred? | Padding, batching, no side-channel timing | | D | Disclosure | Can personal data be exposed? | Access control, encryption, minimization, retention limits | | U | Unawareness | Are users unaware of processing? | Notice, consent, transparency, data-subject access | | N | Non-compliance | Does processing violate law or policy? | Data mapping, retention schedules, DPIAs, deletion paths |

Privacy engineering rules: minimize collection, separate purposes, keep retention short and enforced by code, make deletion a tested path, and treat embeddings and telemetry as personal data when they can be re-identified.

Risk scoring

Scoring exists to order work, not to be precise. Pick one primary scale and stay consistent.

| Tool | Use | Notes | |---|---|---| | CVSS 4.0 | Vulnerabilities with a known technical impact | Base score for triage; threat/environmental metrics only with data | | EPSS | Likelihood a vulnerability is exploited in the wild | Good tie-breaker; low EPSS never excuses a critical on an exposed asset | | SSVC | Decision-oriented triage (track, track*, attend, act) | Maps exploit status and exposure to action; favored by product security teams | | OWASP Risk Rating | Application findings without a CVE | Likelihood and impact factors, useful for bug-bounty triage | | DREAD | Legacy | Deprecated in most programs; avoid for new work |

Combine signals rather than trusting one: exploitability (EPSS, public exploit), exposure (internet, auth required), asset value (data class, blast radius), and detectability. A useful default policy:

• Act now: internet-exposed, exploitable, high asset value, or active exploitation.
• Next cycle: exploitable but not exposed, or exposed with strong compensating controls.
• Track: theoretical, high attack cost, or no path from the entry point.
• Accept: documented, owned, with expiry and a monitoring signal.

Integrating at design time

• Add a threat-model gate to the design review: no model, no approval for boundary-changing work.
• Keep the model as code where possible (Threagile, pytm-style DSLs, Threat Dragon files) so it diffs and reviews like the rest of the repo.
• Export mitigations into the same tracker as functional requirements; label them with the threat id.
• Write abuse cases next to user stories: "as an attacker, I want to replay the webhook so that I can double-credit the account." Each abuse case gets a test.
• Reuse known threat libraries: MITRE ATT&CK for adversary behavior, CAPEC for attack patterns, CWE for weakness classes, cloud provider threat matrices for platform-specific paths.
• Record decisions and accepted risks as ADRs with owner and expiry; an accepted risk with no expiry is an unowned risk.

Anti-patterns

• Threat modeling once, in a wiki, never updated; the next diagram is drawn by an attacker.
• Listing generic threats ("SQL injection") without the specific abuse path through this system.
• Modeling only the happy path and ignoring operational interfaces: admin panels, webhooks, imports, backups, CI.
• Treating the model as a checklist for the security team instead of the design team.
• Scoring every threat with a heavy framework so the exercise never finishes.
• Ignoring privacy because "we are B2B"; B2B data is still personal data.
• Forgetting the AI surface: prompts, retrieval corpora, tool calls, and model providers are new data flows and trust boundaries (see ./08-llm-security.md).
• Mitigations that live only in the document and never become code, tests, or backlog items.

Checklist

• [ ] Diagram covers users, services, data stores, queues, third parties, admin paths, and AI components.
• [ ] Every trust boundary is labeled with protocol and data classification.
• [ ] STRIDE applied per element; each threat has a written abuse path.
• [ ] LINDDUN applied where personal data or telemetry is processed.
• [ ] High-value assets have attack trees; cheapest path addressed.
• [ ] Each threat maps to a control, requirement, test, or owned exception with expiry.
• [ ] Scoring method chosen and applied consistently; exposure and exploitability considered.
• [ ] Model stored in version control and reviewed on boundary changes.
• [ ] Abuse cases added to the test suite.
• [ ] Follow-up session scheduled for the next significant change.

OWASP

Scope: using OWASP artifacts as a requirements and triage backbone — Top 10, ASVS, API Security Top 10, LLM Top 10, and mapping findings to verifiable controls.

How to use OWASP artifacts

OWASP produces several families of documents with different jobs. Do not treat the Top 10 as a standard; treat it as awareness and prioritization. Use ASVS as the requirement catalog, the API Security Top 10 for API programs, and the LLM Top 10 for AI features.

| Artifact | Job | Audience | |---|---|---| | Top 10 | Awareness and risk framing | Everyone, especially product and management | | ASVS | Verifiable security requirements by level | Engineers, testers, architects, assessors | | API Security Top 10 | API-specific risk framing | API and platform teams | | LLM Top 10 | AI feature risk framing | AI/ML and application teams | | Testing Guide (WSTG) | How to verify web controls manually | Testers, red teams | | Cheat Sheet Series | Implementation guidance per topic | Engineers | | SAMM | Program maturity model | Security leadership |

OWASP refreshes the Top 10 on a multi-year cycle; verify the current edition upstream before citing categories. The 2025 edition reorganized several 2021 entries (see the mapping below). ASVS is on a 5.x line; verify the current chapter set upstream.

OWASP Top 10

| # | Risk | What it looks like | Primary controls | |---|---|---|---| | A01 | Broken Access Control | IDOR, missing function-level checks, forced browsing | Deny by default, centralized policy decision, object-level checks, server-side enforcement | | A02 | Security Misconfiguration | Default accounts, verbose errors, open buckets, permissive CORS | Hardened baselines, config as code, environment separation, minimal features, no stack traces to users | | A03 | Software Supply Chain Failures | Malicious or compromised dependency, tampered build, unsigned artifact | SBOM, provenance, signing, pinned deps, see ./06-supply-chain.md | | A04 | Cryptographic Failures | Plaintext transport/storage, weak hashing, hardcoded keys | TLS everywhere, AEAD, Argon2id, managed keys, see ./07-crypto.md | | A05 | Injection | SQL/NoSQL/OS/LDAP injection, template injection (SSTI), XSS as edge case | Parameterized queries, typed APIs, input validation, context-aware output encoding | | A06 | Insecure Design | No threat model, unbounded trust, missing abuse controls | Threat modeling at design time, secure patterns, requirement-level abuse cases, see ./01-threat-modeling.md | | A07 | Authentication Failures | Credential stuffing, weak reset flows, session fixation | MFA/passkeys, rate limiting, secure session lifecycle, see ./03-authn-authz.md | | A08 | Software and Data Integrity Failures | Unsigned updates, deserialization of untrusted data, CI tampering | Signed artifacts, safe serializers, integrity checks, protected CI, see ./06-supply-chain.md | | A09 | Logging and Alerting Failures | No audit trail, unmonitored auth events, logs tamperable | Structured event logs, central pipeline, alerts on security events, see ./09-incident-response.md | | A10 | Mishandling of Exceptional Conditions | Failing open, unhandled errors leaking state, resource exhaustion on error paths | Fail-closed defaults, typed errors, safe error responses, bounded retries and resources |

2021 to 2025 mapping (orientation only)

| 2021 | 2025 direction | |---|---| | A01 Broken Access Control | A01 Broken Access Control (stable) | | A02 Cryptographic Failures | A04 Cryptographic Failures | | A03 Injection | A05 Injection | | A04 Insecure Design | A06 Insecure Design | | A05 Security Misconfiguration | A02 Security Misconfiguration (rank up) | | A06 Vulnerable and Outdated Components | A03 Software Supply Chain Failures (broadened) | | A07 Identification and Authentication Failures | A07 Authentication Failures | | A08 Software and Data Integrity Failures | A08 Software and Data Integrity Failures | | A09 Security Logging and Monitoring Failures | A09 Logging and Alerting Failures | | A10 SSRF | Folded into access control and out-of-band request guidance | | — | A10 Mishandling of Exceptional Conditions (new) |

Do not re-map a program wholesale just because ranks move; use the change list to update requirements and test coverage where categories gained or lost scope.

ASVS levels

ASVS states requirements as verifiable statements organized by chapter, each tagged with a level. Levels are cumulative.

| Level | Meaning | When to target | |---|---|---| | L1 | Baseline, mostly testable without deep expertise | Every application, including internal | | L2 | Defense in depth for applications handling sensitive data | Most business applications, customer data, regulated data | | L3 | High assurance, resistant to advanced threats | Critical infrastructure, health, finance, high-value transactions |

Practical adoption:

1. Pick the target level per application from data classification and exposure, not from ambition.
2. Turn the chosen requirements into tickets and test cases; ASVS is testable by design.
3. Start with the authentication, session management, access control, validation, and crypto chapters; they remove the most risk per unit of effort.
4. Record out-of-scope requirements and L3 gaps as owned exceptions.
5. Reassess when exposure or data class changes; a tool that becomes internet-facing jumps to L2/L3 obligations.

Common chapter themes: architecture and threat modeling, authentication, session management, access control, input validation and encoding, stored cryptography, error handling and logging, data protection, communications, malicious code and supply chain, business logic, file handling, API and web service, configuration, and (in 5.x) data layer and application security posture themes. Verify the exact chapter list upstream.

API Security Top 10

| # | Risk | Example | Control | |---|---|---|---| | API1 | Broken Object Level Authorization | GET /orders/123 returns another user's order | Per-object authorization on every access | | API2 | Broken Authentication | Weak token issuance, no revocation, credential stuffing | Strong authn, short tokens, rotation, rate limits | | API3 | Broken Object Property Level Authorization | Mass assignment or excessive data exposure | Explicit allowlists for writable and readable fields | | API4 | Unrestricted Resource Consumption | No pagination/limits; billable third-party calls | Quotas, pagination caps, timeouts, cost limits | | API5 | Broken Function Level Authorization | Regular user reaches admin endpoint | Function-level policy, deny by default, no hidden routes | | API6 | Unrestricted Access to Sensitive Business Flows | Scalping, spam, abuse of a legitimate flow | Flow-specific controls, velocity limits, human checks | | API7 | Server-Side Request Forgery | URL parameter fetches internal metadata | Egress allowlist, no user-controlled egress, metadata protections | | API8 | Security Misconfiguration | Verbose errors, permissive CORS, missing headers | Hardened gateway config, consistent error contract | | API9 | Improper Inventory Management | Shadow and deprecated API versions exposed | Version registry, deprecation policy, gateway inventory | | API10 | Unsafe Consumption of APIs | Trusting upstream API data blindly | Validate and sanitize third-party responses, TLS pinning where warranted |

API design specifics belong in an API-design pack; this reference only fixes the risk-to-control mapping. See ./03-authn-authz.md for token and scope handling and ./04-security-architecture.md for gateway and policy placement.

LLM Top 10

The LLM list tracks AI feature risks; full treatment is in ./08-llm-security.md.

| # | Risk | One-line control | |---|---|---| | LLM01 | Prompt Injection | Treat model input/output as untrusted; constrain privileges and tools | | LLM02 | Sensitive Information Disclosure | Minimize context, filter outputs, tenant-isolate retrieval | | LLM03 | Supply Chain | Vet models, datasets, plugins, and serving stacks; attest artifacts | | LLM04 | Data and Model Poisoning | Provenance, curation, anomaly detection on training/retrieval data | | LLM05 | Improper Output Handling | Validate and encode before any downstream sink | | LLM06 | Excessive Agency | Least-privilege tools, human approval, bounded autonomy | | LLM07 | System Prompt Leakage | No secrets in prompts; assume prompts are public | | LLM08 | Vector and Embedding Weaknesses | Access control at retrieval, embedding hygiene, poisoning checks | | LLM09 | Misinformation | Grounding, citations, confidence surfacing, human review for high stakes | | LLM10 | Unbounded Consumption | Quotas, token and cost caps, timeouts, abuse monitoring |

An Agentic Security Initiative body of work extends this to autonomous agent risks; verify its current status upstream and load it when agents hold credentials or act without a human in the loop.

Mapping findings to controls

Use one pipeline for scanner output, pen-test reports, and bug bounties so fixes are comparable and trackable.

1. Reproduce and scope. Confirm the finding against a running artifact; note entry point, authentication required, and affected data.
2. Classify. Map to the OWASP category and the underlying CWE (for example, IDOR to CWE-639 and API1/A01). One finding may map to several categories; pick the controlling one.
3. Locate the requirement. Find the ASVS requirement(s) that the control must satisfy; the text becomes the acceptance criterion.
4. Choose the fix. Prefer a structural control (central authz, parameterized API) over a spot patch (one extra if).
5. Add a test. Every fix gets a regression test that fails on the old code; for authz, test the negative case across roles and tenants.
6. Track to closure. Owner, severity, SLA, and exception path; reopen automatically if the test regresses.
7. Report upward. Roll up by category and business unit; count open criticals and mean age, not raw finding volume.

| Finding source | Typical categories | Verification after fix | |---|---|---| | SAST | Injection, crypto misuse, hardcoded secrets | Unit/integration tests, re-scan | | DAST | Authz gaps, misconfig, injection | Authenticated regression tests | | SCA | Known-vulnerable dependencies | Rebuild and redeploy, reachability analysis | | IaC scan | Cloud/K8s misconfig | Policy-as-code gates, drift detection | | Pen test | Business logic, chained access | Abuse-case tests, control redesign | | LLM red team | Injection, exfiltration, tool abuse | Adversarial eval suite, see ./08-llm-security.md |

Verification tooling

| Layer | Tools (examples) | Cadence | |---|---|---| | SAST | Semgrep, CodeQL, language linters | Every PR, diff-scoped plus nightly full | | DAST | ZAP, Burp (automation), API fuzzers | Nightly against staging, on release | | SCA/dependencies | OSV-based scanners, Dependabot, Renovate | Every PR, nightly, continuously on running artifacts | | Secrets | gitleaks, trufflehog, provider scanning | Pre-commit, PR, continuous | | IaC/K8s | Checkov, Trivy config, kube-linter, Kyverno | Every PR, admission time | | LLM | garak, PyRIT, promptfoo-style evals | On model/prompt changes, scheduled | | Manual | WSTG-based test cases, threat-model-driven abuse cases | Per release for high-risk changes |

Automation finds classes; humans find chains. Combine scanner gates with a threat-model-driven manual pass on the highest-risk flows.

Anti-patterns

• Quoting "OWASP compliant" as a property; compliance is against ASVS-style verifiable requirements.
• Mapping every finding to A01 and calling the report complete.
• Running SAST at maximum noise and disabling the gate because of false positives instead of tuning rules.
• Treating the Top 10 as the test plan and skipping the threat model.
• Fixing the symptom (one endpoint) without the class-wide control (policy layer).
• Letting scanner findings sit in a separate tracker from engineering work.
• Applying the LLM list only to the model and not to tools, retrieval, and outputs.
• Assuming internal APIs need no authz or rate limits.

Checklist

• [ ] Target ASVS level chosen per application and recorded.
• [ ] Chosen ASVS requirements mapped to tickets and tests.
• [ ] Top 10 categories reviewed against the current design, with owners for gaps.
• [ ] API endpoints checked against all ten API risks, including object- and property-level authz.
• [ ] LLM features checked against the LLM risks and reviewed with ./08-llm-security.md.
• [ ] Finding-to-control pipeline defined with severity SLAs and exception process.
• [ ] SAST, DAST, SCA, secrets, and IaC scanning wired into CI with tuned gates.
• [ ] Regression tests exist for every fixed class of finding.
• [ ] Metrics report open criticals and mean age, not raw counts.
• [ ] Current OWASP edition and chapter sets verified upstream at review time.

Authentication and Authorization

Scope: identity for humans, services, and agents — OAuth 2.1, OIDC, WebAuthn/passkeys, session management, MFA, authorization models, and service-to-service auth.

Model the problem first

Separate three questions. Conflating them is the root of most identity bugs.

1. Authentication — who is this principal, and how strongly was that proven?
2. Session/token — how does that proof persist across requests, and how is it revoked?
3. Authorization — may this principal perform this action on this object, in this context?

Also separate principal types: human interactive, human via third-party app, service/workload, and autonomous agent. Each has different credential lifetimes and revocation paths.

Protocol selection

| Situation | Use | Never | |---|---|---| | First-party web app, server-rendered or BFF | OIDC authorization code with PKCE and server-side session cookie | Tokens in localStorage | | SPA or mobile app calling your API | Authorization code with PKCE; BFF pattern when possible | Implicit flow, password grant | | Third-party access to your API | OAuth 2.1 authorization code with PKCE, scoped tokens, consent | Shared API keys handed to partners | | Service to service in one trust domain | Workload identity with mTLS (SPIFFE-style), short-lived tokens | Static API keys in config | | Service to service across domains | OAuth client credentials with mTLS/private key JWT, or token exchange | Long-lived shared secrets | | CLI, TV, IoT with limited input | Device authorization grant | Embedding a client secret in the binary | | Enterprise workforce SSO | OIDC (preferred) or SAML 2.0 | Home-grown directory federation | | High-value consumer access | Passkeys (WebAuthn) with recovery design | SMS as the only second factor |

OAuth 2.1 consolidates the security best current practice: PKCE required for all authorization code flows, implicit and password grants removed, exact redirect URI matching, and rotation for public clients. Verify the RFC number and status upstream; the safe engineering posture (PKCE everywhere, no implicit) is already mandatory.

OAuth 2.1 and OIDC essentials

Authorization code with PKCE:

1. Client generates code_verifier, sends code_challenge (S256) with the authorization request, plus state and a nonce for OIDC.
2. User authenticates at the IdP; the IdP redirects back with a code to an exactly matched redirect URI.
3. Client exchanges the code plus code_verifier at the token endpoint (authenticated for confidential clients).
4. Client validates tokens and creates its own session.

Rules:

• Verify state (CSRF), nonce (replay), and exact redirect URI match.
• Request the narrowest scopes; scopes are coarse gates, claims are data. Never encode authorization decisions only in scopes.
• Access tokens are for resource servers, not for the browser. Keep them in the backend when a BFF is available.
• Prefer sender-constrained tokens where the threat model includes token theft: DPoP or mTLS-bound tokens.
• Use audience-restricted tokens; a token for service A must not work at service B. Reject tokens with the wrong aud or issuer.
• Rotate refresh tokens and detect reuse; a reused refresh token invalidates the family.

OIDC ID token validation, in order:

1. Signature against the issuer's JWKS, with key rotation handled by kid lookup and caching.
2. iss equals the expected issuer (exact string match).
3. aud contains this client, and azp matches when multiple audiences exist.
4. exp/iat/nbf within acceptable clock skew; reject expired or far-future tokens.
5. nonce matches the one you sent.
6. Only then read claims (sub, email_verified, groups) and map to local identity.

SAML notes: validate signatures over the assertion and the response with an allowlisted key, defend against XML signature wrapping, enforce Audience, Recipient, InResponseTo, and NotOnOrAfter, and prefer OIDC for new integrations.

WebAuthn and passkeys

WebAuthn gives phishing-resistant authentication; passkeys are discoverable credentials synced or device-bound by the platform.

Registration (navigator.credentials.create):

• Server sends a random challenge, RP id, user handle, and a list of acceptable algorithms.
• Verify the attestation as far as policy requires (most consumer services use none; high assurance may require specific attesting keys).
• Store credential id, public key, sign counter, transports, and AAGUID; never a biometric or PIN.

Authentication (navigator.credentials.get):

• Server sends a random challenge and allowed credentials; verify the signature over authenticator data plus client data, check rpIdHash, user presence/verification flags, and UV for sensitive operations.
• Treat counters as a signal, not a hard lock; synced passkeys may not increase them.
• Require user verification for step-up actions such as adding a key or changing recovery methods.

Operational rules:

• Offer passkeys first; keep TOTP or another fallback for compatibility, but not SMS-only.
• Design recovery deliberately — recovery is the weakest link in every passkey deployment. Use multiple registered keys, recovery codes stored offline, or an assisted identity-proofing flow; never an email reset that downgrades to a single factor.
• Support account recovery without exposing credential enumeration: identical responses for known and unknown users.
• Bind passkey registration and deletion to a re-authenticated session to prevent credential injection from a hijacked session.

Session management

| Concern | Practice | |---|---| | Cookie flags | Secure, HttpOnly, SameSite=Lax or Strict; __Host- prefix for session cookies | | Session id | At least 128 bits of CSPRNG entropy; opaque, not derived from user data | | Rotation | New id on login, privilege change, and periodic intervals; invalidate the old id | | Lifetimes | Short idle timeout for sensitive apps, absolute maximum regardless of activity | | Revocation | Server-side session store; revoke on logout, password change, and credential compromise | | Concurrent sessions | Show and allow revoking active sessions; cap where policy requires | | Binding | Consider binding to a device or token where risk warrants; handle false positives | | CSRF | Synchronizer token or double-submit for cookie-based flows, Origin checks as defense in depth | | Logout | Clear server state, expire cookies, and call IdP logout where single logout is required |

For bearer-token APIs with no browser session, treat refresh tokens as the crown jewel: store them hashed, rotate on use, bind to the client where possible, and support revocation per device.

MFA

| Factor | Strength | Notes | |---|---|---| | Passkey/WebAuthn | Phishing-resistant | Preferred for admins and high-value accounts | | Hardware security key | Phishing-resistant | Same WebAuthn mechanics, portable assurance | | TOTP app | Moderate | Shared secret; phishable in real time; still valuable | | Push with number matching | Moderate | Add number matching or origin binding; avoid simple approve/deny | | SMS/voice OTP | Weak | SIM swap and interception; fallback only, with monitoring | | Email OTP | Weak | Acceptable only for low-risk recovery with rate limits | | Security questions | Very weak | Avoid; often publicly discoverable |

Program rules: enforce MFA for all privileged roles and all remote access; register at least two factors; use step-up authentication for sensitive actions (changing credentials, exporting data, approving payments); throttle and alert on failed factor attempts; never allow MFA setup to bypass re-authentication.

JWT pitfalls

| Pitfall | Fix | |---|---| | alg confusion or none accepted | Pin the expected algorithm; reject unknown alg | | Signature verified but issuer/audience ignored | Validate iss, aud, exp, nbf every time | | Symmetric secret shared across services | Use asymmetric keys or mTLS-bound tokens; never HS256 with a public client | | Long-lived tokens | Minutes for access tokens, rotation for refresh | | Sensitive data in claims | JWT payloads are readable; keep PII and secrets out | | No revocation story | Short expiry plus denylist or introspection for high-value operations | | kid header trusted blindly | Resolve keys from a trusted JWKS by kid, not from the token itself | | Clock skew ignored or excessive | Small skew (tens of seconds), monitor expiry errors |

Authorization models

| Model | Semantics | Strengths | Costs | |---|---|---|---| | RBAC | Roles grant permissions | Simple, auditable, easy to reason about | Role explosion; poor fit for per-object sharing | | ABAC | Policies over attributes and context | Precise, contextual (time, device, data class) | Policy complexity, attribute quality risk | | ReBAC | Relationships define access (Zanzibar-style) | Natural for sharing and hierarchies; scalable checks | Requires relationship store and model discipline | | Hybrid | ReBAC for objects plus ABAC constraints | Handles multi-tenant SaaS with sharing | Two systems to operate; keep one policy decision point |

Enforcement rules that hold for every model:

• One decision point. Centralize policy evaluation in a service or sidecar; applications ask, policy decides. Avoid per-endpoint ad hoc logic.
• Object-level checks always. Route-level checks are not authorization. Fetch the object, then ask the policy engine whether this subject may act on this object.
• Deny by default. Unknown action, missing attribute, or policy service error means deny (fail closed), with a monitored error path.
• Use list-and-filter for collections. Do not fetch all rows then filter in memory; push authorization into the query or use relationship lookups.
• Model tenancy explicitly. Tenant id is part of the resource identity and a mandatory input to policy, never an optional header.
• Separate admin planes. Privileged actions have separate roles, separate authn strength, and full audit.
• Keep policy in version control, tested. Policy changes are code changes with tests and review; changes are the highest-risk deploys.

Service-to-service authentication

• Give every workload a cryptographic identity: SPIFFE IDs or cloud workload identity. mTLS provides authentication and encryption in transit; authorization still needs an explicit policy on top.
• Prefer platform-issued, short-lived credentials (certificates or tokens) over static API keys. If an API key is unavoidable, hash it at rest, prefix it for identification, scope it narrowly, and rotate it.
• Use the token exchange pattern when a service must call downstream on behalf of a user; preserve sub and act so downstream authorization can see both the user and the actor.
• Authenticate callers at the mesh or gateway and pass identity in a signed, verifiable form (mTLS identity, signed headers). Never trust unsigned internal headers such as X-User-Id.
• Network location is not authentication: "inside the VPC" is not a principal.

Agent and machine identities

• Give each agent its own identity and scopes; never lend it a human's broad credentials.
• Scope tools per task and per tenant; default to read-only, require approval for writes to high-impact systems.
• Make agent actions attributable in logs: agent id, on-behalf-of user, tool, input summary, result.
• Support kill-switch revocation per agent and per tool grant.

Anti-patterns

• Home-grown session tokens or password crypto.
• Checking auth != null instead of can(subject, action, object).
• Passing user id in a header or query parameter and trusting it.
• Storing tokens in localStorage and calling it a session strategy.
• Using one long-lived API key for many services or customers.
• Roles named after people, or roles that silently accumulate permissions.
• MFA that can be disabled by email-only reset.
• Trusting internal network traffic; no mTLS, no identity, no authz.
• Policy logic duplicated in three languages and diverging.
• No revocation path: the only way to remove access is waiting for expiry.

Checklist

• [ ] Authn mechanism chosen from the protocol table, with PKCE and exact redirect matching where OAuth applies.
• [ ] ID/access tokens fully validated (alg, signature, iss, aud, exp, nonce).
• [ ] Sessions use secure cookie attributes, rotation, idle and absolute timeouts, and server-side revocation.
• [ ] MFA enforced for privileged access; at least two strong factors registered.
• [ ] Passkey recovery designed and tested; no single-factor downgrade.
• [ ] Authorization centralized, object-level, deny-by-default, and tested for negative cases.
• [ ] Tenancy enforced in policy inputs and data access, not in presentation code.
• [ ] Service identities use mTLS or workload identity with scoped, short-lived credentials.
• [ ] Agents have separate identities, scoped tools, approval gates, and revocation.
• [ ] Auth events and policy decisions logged to a central, tamper-resistant store.

Security Architecture

Scope: system-level security design — defense in depth, zero trust, segmentation, secure defaults, trust boundaries, key management architecture, and architecture review.

Principles

| Principle | Meaning | Design consequence | |---|---|---| | Defense in depth | Independent layers so one failure is not a breach | Authn plus authz plus validation plus monitoring on the same path | | Least privilege | Minimum rights, shortest duration | Scoped roles, short-lived credentials, no ambient authority | | Complete mediation | Every access is checked, every time | Central policy decision point; no cached "already checked" assumptions | | Fail secure | Failure denies access and preserves integrity | Closed-by-default gateways, deny on policy-engine errors | | Economy of mechanism | Simple, reviewable designs | Fewer trust paths, no bespoke protocols, small policy surface | | Separation of duties | No single principal controls a critical operation end to end | Dual control for key ceremonies, production changes, and payments | | Least common mechanism | Shared components do not create shared risk | Tenant isolation, separate control and data planes | | Open design | Security does not depend on secrecy of design | Public algorithms; secrets limited to keys | | Minimize attack surface | Fewer ports, features, and dependencies | Disable unused services; remove debug and admin endpoints from public routes | | Secure by default | The safe configuration is the zero-effort configuration | Deny policies on until explicitly opened; TLS on by default |

Trust boundaries

A trust boundary is anywhere control, identity, or data classification changes: browser to edge, edge to cluster, service to service across namespaces, application to database, workload to cloud metadata, service to a third-party API, and application to an LLM or agent tool. Find them from the data-flow model in ./01-threat-modeling.md.

For each boundary, define:

1. Enforcement — what authenticates and authorizes crossing it (gateway, mesh mTLS, policy engine, token exchange).
2. Validation — what schema and semantic checks run on entry.
3. Data handling — what may cross, in what form (classification, redaction, field-level rules).
4. Observability — what is logged and alerted at the crossing.
5. Failure mode — what happens when the enforcement point is unavailable (deny, queue, degraded read-only).

Rule of thumb: crossing a boundary requires an explicit decision; crossing inside a boundary may inherit context, but never inherits trust to a different classification.

Zero trust

Zero trust replaces network location with explicit, continuously evaluated identity and context. NIST SP 800-207 is the canonical reference (cloud-native guidance in SP 800-207A); verify the current revision upstream.

Core tenets in practice:

• Every request is authenticated and authorized based on identity, device, and context, regardless of network.
• Access is per-session with least privilege and just-in-time elevation, not standing network access.
• Policy is centralized and dynamic; enforcement is distributed to gateways, proxies, and sidecars.
• Assume the network is hostile: encrypt and authenticate service traffic, verify endpoints.
• Log everything relevant to access decisions for detection and improvement.

Architecture building blocks:

| Component | Role | Examples | |---|---|---| | Policy decision point (PDP) | Evaluates policy on identity and context | OPA, Cedar, cloud policy engines | | Policy enforcement point (PEP) | Applies the decision at the data path | API gateway, sidecar, service mesh, IdP-aware proxy | | Identity provider | Strong authentication and token issuance | OIDC IdP with passkeys and MFA | | Device posture | Device health and management signal | MDM signal, certificate-based device identity | | Identity-aware proxy | Replaces VPN for workforce app access | Provider ZTNA solution or self-built gateway with OIDC | | Continuous evaluation | Re-checks sessions on context change | Short sessions, re-auth on risk signals, revocation feeds |

Migration reality: you cannot buy zero trust in one product. Sequence: strong identity, then segmentation and mTLS, then centralized policy, then device signals and continuous evaluation.

Segmentation

• Place workloads in tiers by data classification and exposure: public edge, application, data, management. Allow only declared flows between tiers; default deny within the cluster as well.
• Enforce L3/L4 network policy as a floor and L7 policy where identity matters (method, path, service identity). See ./05-container-cloud-security.md for Kubernetes specifics.
• Isolate management planes (CI, bastion, admin APIs) from data planes; management never rides on the same path as user traffic.
• Egress is a boundary too: allowlist destinations for workloads, especially anything that processes untrusted input or hosts an agent.
• Deny lateral movement by default: no flat RFC1918 trust, no shared credentials between tiers, no "internal" header trust.
• Use separate accounts/projects/subscriptions per environment and blast-radius zone; production credentials do not exist in dev.

Secure defaults

| Area | Safe default | Common unsafe default | |---|---|---| | Network | Deny all ingress and egress; open explicitly | Allow-all then restrict later | | Authz | Deny when policy is missing or evaluation fails | Allow when no rule matches | | TLS | On, valid certs required, no skip-verify option in prod | Optional TLS, insecure flags wired for convenience | | Debug/admin | Disabled in production images and routes | Debug endpoints reachable from the internet | | CORS | No wildcard with credentials; explicit origins | * plus credentials | | Cookies | Secure, HttpOnly, SameSite, prefixed | No attributes, long-lived | | Storage | Private buckets, encryption on, access logging | Public buckets, no logging | | Errors | Generic external messages, detailed internal logs | Stack traces and queries returned to clients | | Feature flags | Default off for risky features; kill switch ready | Default on, no off switch | | Secrets | Required from a manager; no fallback literal | "Temporary" hardcoded key that survives |

Make the safe path the easy path: scaffolding, templates, and CI checks should generate the secure configuration so developers do not choose it manually.

Key management architecture

A key hierarchy limits exposure: root keys never encrypt data directly.

| Layer | Function | Storage | Rotation | |---|---|---|---| | Root of trust / CMK | Wraps key-encryption keys; root of the hierarchy | HSM or KMS, never exported | Rare, under dual control and ceremony | | Key encryption key (KEK) | Wraps data keys; may be per-tenant | KMS or HSM-backed | On schedule and on compromise | | Data encryption key (DEK) | Encrypts data with AEAD | Stored encrypted (envelope) or ephemeral in memory | Per object, per message, or per short period | | Application secrets | Passwords, API credentials, tokens | Secret manager with workload identity | Automated, short-lived where possible |

Design rules:

• Never ship root or KEK material into application memory; use KMS/HSM wrap/unwrap or envelope encryption with DEKs.
• Bind ciphertext to context: store key id/version and encryption metadata with the data; support decrypt-with-old-key while writing new keys.
• Plan rotation before you need it: every ciphertext carries enough metadata to rewrap; run a rotation drill in staging.
• Enforce separation of duties: key administrators cannot read data encrypted under the keys they manage; use quorum or dual control for destructive key operations.
• Document the crypto boundary: which components can see plaintext, which only ciphertext, and which hold key material. Data must be plaintext only inside the boundary.
• BYOK/HYOK changes trust in the provider, not your key hygiene; verify provider semantics (import, rewrap, revocation, regional residency) before relying on them.
• Prefer per-tenant keys for isolation, but track KMS quotas and cost; quota exhaustion is an availability risk.
• Capture an inventory: key id, purpose, algorithm, owner, creation, rotation schedule, and dependent systems. Unknown keys cannot be rotated or revoked.

Pattern catalog

| Problem | Pattern | Notes | |---|---|---| | Browser app calling APIs | Backend for frontend (BFF) with server-side session | Keeps tokens off the browser; see ./03-authn-authz.md | | Cross-cutting authz | Policy sidecar or library with central PDP | One decision semantics; policy tested as code | | Third-party integration | Gateway with schema validation and egress allowlist | Boundary control in one place | | High-risk actions | Step-up authentication plus dual approval | Risk-based challenge on the action, not only on login | | Untrusted code execution | Sandboxed worker with no credentials and bounded egress | Also applies to agent code tools | | Multi-tenant SaaS | Tenant id in the resource identity; per-tenant keys and policy scope | Isolation enforced in data access, not UI | | Secrets delivery | CSI/secret operator with workload identity | No plaintext secrets in manifests or images | | LLM feature | Tool gateway with allowlisted actions and human approval for writes | See ./08-llm-security.md |

Security architecture review

Inputs: data-flow model and threat list, data classification, compliance obligations, deployment topology, identity model, and the ADRs for prior decisions.

Review outputs:

1. Boundary table (enforcement, validation, data handling, observability, failure mode) as above.
2. Control placement map: which layer owns each control and what happens when it fails.
3. Open risks with owner, severity, and expiry.
4. Security decisions recorded as ADRs, linked from the system's docs.
5. Follow-up items with tests or verification steps.

Review questions that catch most design flaws:

• What is the worst thing an authenticated user can do with one request?
• What happens when each external dependency fails or is compromised?
• Where is plaintext data, and who can reach it?
• Which credentials exist, how long do they live, and how are they revoked?
• What happens to existing sessions and tokens when an account is disabled?
• Can a single compromised CI job or admin account reach production data?
• Which paths cross an autonomy boundary (an agent acting without approval)?

Anti-patterns

• Security as a perimeter: hard shell, soft center.
• Copying a reference architecture without its operating assumptions (scale, tenancy, compliance).
• One giant shared VPC/subscription with allow-all security groups.
• Encryption without key management: keys in config, one key for everything, no rotation.
• Adding a control in the UI and calling it enforced.
• Policy evaluation that fails open when the PDP is down.
• Rebuilding zero trust as a VPN replacement only, with no per-request authorization.
• Threat model updated only when an auditor asks.
• Admin plane colocated with the user data plane on the same credentials.

Checklist

• [ ] Data-flow model identifies boundaries; each boundary has enforcement, validation, data rules, observability, and a failure mode.
• [ ] Zero-trust rollout sequenced: identity first, then segmentation, policy, device, continuous evaluation.
• [ ] Network denies by default for ingress and egress; management planes isolated.
• [ ] Secure defaults documented and generated by templates; unsafe options require explicit override.
• [ ] Key hierarchy documented with roles, storage, rotation, and separation of duties.
• [ ] Every ciphertext carries key id and metadata; rotation drill executed.
• [ ] Crypto boundary and plaintext visibility documented.
• [ ] Patterns chosen deliberately: BFF, PDP/PEP, gateway, sandbox, per-tenant isolation.
• [ ] Security review performed for boundary-changing work; decisions recorded as ADRs.
• [ ] Exceptions have owners and expiry; controls have owners and tests.

Container and Cloud Security

Scope: hardening Kubernetes, images, admission, cloud IAM, network policy, cloud secrets, and continuous posture management.

Shared responsibility

Cloud providers secure the provider plane; you secure identities, configuration, workloads, and data. Every service has a different split, so write down the boundary per service. Assume misconfiguration, not provider compromise, is the primary threat.

| Layer | You own | Provider owns | |---|---|---| | Cloud IAM | Policies, roles, federation, credential hygiene | IAM service availability and integrity | | Managed Kubernetes | Workloads, RBAC, network policy, admission, images | Control plane availability (in managed offerings) | | VMs and containers | Patching, hardening, runtime config | Hypervisor and physical | | Managed data services | Access control, encryption config, backups, schema | Engine, patching of managed versions | | Serverless | Code, identity, triggers, dependencies | Runtime and platform scaling |

Kubernetes hardening baseline

| Area | Practice | |---|---| | Versions | Run supported minors only (the project maintains roughly the latest three; verify upstream) and patch promptly | | API server | Private endpoint where possible, strong authn, audit logging on, authorization via RBAC, disable anonymous auth | | etcd | Encryption at rest for secrets, TLS, restricted network access, tested backups | | RBAC | Namespaced roles, no cluster-admin for applications, no wildcard verbs/resources, audit role bindings | | Service accounts | Disable token automount unless needed; projected tokens with audience and expiry; one SA per workload | | Pod Security | Pod Security Admission restricted for namespaces by default; exceptions documented and narrowed | | Workload context | runAsNonRoot, fixed high UID, read-only root filesystem, drop ALL capabilities, allowPrivilegeEscalation: false, seccompProfile: RuntimeDefault | | Host access | No hostPath, hostNetwork, hostPID, or privileged containers without reviewed exceptions | | Network | Default-deny policy; explicit ingress and egress; see below | | Secrets | External secret manager or encrypted store; never plain Secrets created by hand | | Admission | Policy that blocks the above violations; see below | | Node access | No SSH by default; use debug pods with explicit, time-boxed grants | | Audit | API audit logs shipped off-cluster; alerts on privileged role changes and exec into pods |

# Pod security-context fragment (baseline)
securityContext:
  runAsNonRoot: true
  runAsUser: 65532
  seccompProfile: { type: RuntimeDefault }
containers:
  - name: app
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities: { drop: ["ALL"] }

Image security

• Use minimal bases: distroless or scratch for static binaries, slim images otherwise. Fewer packages mean fewer vulnerabilities and a smaller attack surface.
• Pin base images by digest; rebuild regularly to pick up patched bases. A pinned digest without a rebuild path ages into vulnerability debt.
• Multi-stage builds: build tools never ship in the runtime image.
• Run as non-root, set a fixed UID, and use a read-only root filesystem with explicit writable mounts.
• Scan images on build and continuously for running digests; gate criticals with a fix available.
• Sign images and attest SBOM/provenance; verify at admission. Details in ./06-supply-chain.md.
• Keep a runtime inventory linked to SBOMs so "which workloads ship component X" is a query.

FROM golang:1.25-alpine AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -o /out/api ./cmd/api

FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/api /api
USER 65532:65532
ENTRYPOINT ["/api"]

(Go toolchain versions move quickly; use a supported release and verify upstream.)

Admission control

Admission is the enforcement point for cluster-wide policy. Prefer built-in mechanisms before adding webhooks, because webhooks add availability risk.

| Mechanism | Use | Notes | |---|---|---| | Pod Security Admission | Namespace-level baseline/restricted | Built in; label namespaces; exceptions per namespace | | ValidatingAdmissionPolicy (CEL) | Custom, in-process validation | No webhook dependency; good for simple invariants | | Kyverno | Policy engine with image verification and mutation | Broad library; validate policies in CI | | Gatekeeper (OPA) | Rego policy with constraint templates | Strong when you already use OPA | | Sigstore policy-controller | Keyless signature verification | Pairs with cosign | | Mutating webhooks | Defaults injection (sidecars, labels) | Keep minimal; each webhook is a cluster availability dependency |

Rules for policy systems:

• Run policies in audit mode first, measure violations, then enforce; never enforce untested policy cluster-wide.
• Test policies against manifests in CI; policy is code with review.
• Fail closed for security checks, but operate webhooks with multiple replicas, failure policies chosen deliberately, and narrow match scopes.
• Log policy denials; frequent denials usually mean defaults are wrong, not that attackers are busy.

Cloud IAM pitfalls

| Pitfall | Why it hurts | Control | |---|---|---| | Wildcard actions/resources | One compromised role owns the account | Least privilege, generated from access logs, reviewed periodically | | Long-lived access keys | Leak once, valid forever | Workload identity, OIDC federation, no static keys | | Confused deputy | A service is tricked into acting for the wrong tenant | External IDs, audience conditions, source-account restrictions | | Cross-account trust with * principal | Any account can assume | Restrict principal, add conditions, monitor trust policy changes | | Privilege escalation paths | Small permissions compose into admin (iam:PassRole, iam:CreatePolicyVersion) | Analyze with CIEM tooling, remove composition paths | | Permissions at the account root | Blast radius equals the account | Separate accounts/projects per environment and function | | Unused identities | Forgotten roles and keys are entry points | Inventory, last-used analysis, automatic disable | | Metadata service exposure | SSRF reaches instance credentials | Require IMDSv2 or equivalent, hop limits, egress blocks | | Over-scoped CI roles | Pipeline can change production without review | Separate plan and apply roles, environment approvals, OIDC with conditions | | Missing guardrails | Nothing blocks obvious misconfig | Permission boundaries, SCPs/organization policies, deny lists |

Identity hygiene: every human uses SSO with MFA; every workload uses a distinct identity; every credential has an owner and an expiry; every cross-boundary trust has conditions.

Network policy

• Start with default-deny ingress and egress per namespace; add explicit flows.
• Policy must cover both directions; an ingress-only policy still allows data exfiltration.
• Egress allowlists are critical for workloads processing untrusted input: agent runtimes, scrapers, renderers, and anything with SSRF potential.
• Use identity-based policy where available (Cilium, Calico with service accounts) rather than IP ranges that change with scaling.
• Watch DNS: pod egress policies that block DNS break everything and get disabled. Allow the cluster DNS explicitly.
• Test policy with connectivity tests in CI or a staging cluster; a policy that blocks a dependency is an outage.
• Consider L7 policy for mesh or eBPF platforms when method/path restrictions add real value; otherwise L3/L4 identity policy is enough.

# Default deny for a namespace (Cilium example; NetworkPolicy works similarly)
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: default-deny, namespace: app }
spec:
  endpointSelector: {}
  ingress: []
  egress:
    - toEndpoints:
        - matchLabels: { k8s:io.kubernetes.pod.namespace: kube-system, k8s-app: kube-dns }
      toPorts:
        - ports: [{ port: "53", protocol: UDP }]

Secrets in cloud

• Source of truth: a managed secret store (cloud secret manager, Vault) or encrypted Git with a sealed-secrets pattern; pick one and document it.
• Delivery: External Secrets Operator or Secrets Store CSI with workload identity. Avoid hand-created Kubernetes Secrets and avoid baking secrets into images.
• Prefer dynamic credentials: database IAM auth, short-lived STS tokens, secretless access to storage via workload identity.
• Encrypt at rest with your own KMS key where supported; track which key protects which secret.
• Rotate automatically and test rotation in staging; an untested rotation procedure is broken.
• Audit secret access and alert on reads from unexpected principals.
• Do not put secrets in environment variables where avoidable: they leak through crash dumps, child processes, and process listings. Prefer files with tight permissions or SDK-side retrieval.

Runtime security and posture

| Program | Question | Typical tooling | |---|---|---| | CSPM | Is the cloud account configured safely? | Provider-native and third-party posture scanners | | CIEM | Who can do what, and what is unused? | Effective-permission analysis, last-used data | | KSPM | Are clusters compliant with the baseline? | CIS benchmark scanners, policy reports | | CWPP | What is running and are workloads protected? | Agent-based and agentless workload scanning | | Runtime detection | Is behavior anomalous now? | Falco, Tetragon, seccomp/AppArmor enforcement | | Vulnerability management | What is exposed and fixable? | Registry scanning plus running-image scanning |

Operating model: scan continuously, route findings to owners, fix by SLA, and track drift. Posture tools that produce reports nobody reads are expensive logging.

Anti-patterns

• Single shared cluster for all environments with cluster-admin for everyone.
• Admission webhooks enforced everywhere with one replica.
• Signed images verified only at build, with mutable tags deployed.
• Long-lived cloud access keys stored in CI secrets.
• aws_iam_policy with Action: "*" and a comment saying "tighten later".
• Default VPC with allow-all security groups and public databases.
• Network policy applied only to the default namespace.
• Kubernetes Secrets created manually and never rotated.
• Runtime agents deployed in detect-only mode forever.
• Patching clusters only when an audit is scheduled.

Checklist

• [ ] Cluster runs supported versions; control plane and etcd hardened; audit logs exported.
• [ ] RBAC follows least privilege; no application cluster-admin; wildcards removed.
• [ ] Pod Security restricted by default with documented exceptions.
• [ ] Workload security contexts set (non-root, read-only, no caps, seccomp).
• [ ] Images minimal, digest-pinned, signed, scanned on build and continuously.
• [ ] Admission policy enforced with tested policies and HA webhooks.
• [ ] Cloud IAM uses workload identity; no long-lived keys; escalations analyzed.
• [ ] Default-deny ingress and egress with explicit flows and tested connectivity.
• [ ] Secrets delivered by identity from a manager; rotation automated and tested.
• [ ] Runtime detection on; posture programs owned with SLAs and drift tracking.

Supply Chain Security

Scope: securing the path from source to running artifact — SBOM, SLSA, sigstore, dependency provenance, build hardening, and vendor risk.

Threat model

| Stage | Threats | Primary controls | |---|---|---| | Source | Compromised developer account, malicious dependency, typo-squatting, self-merging PR | SSO with MFA, branch protection, required review, lockfiles, dependency review | | Build | Poisoned CI, tampered steps, credential theft, cache poisoning | Ephemeral runners, OIDC, isolated builds, pinned actions, provenance | | Artifact | Registry tampering, tag mutation, unsigned or downgraded image | Digests, signing, registry immutability, admission verification | | Distribution | Mirror compromise, dependency confusion | Scoped registries, allowlisted indexes, signature verification | | Deploy | Unauthorized artifact, policy bypass, secret exposure in pipeline | Admission policy, least-privilege deploy identity, secret managers | | Runtime | Exploited vulnerable dependency, malicious package at runtime | Continuous scanning, rebuild cadence, runtime detection |

The goal is not zero vulnerabilities; it is that every deployed artifact traces to a reviewed source commit and a trusted build, and that you can answer impact questions in minutes.

SBOM

Generate a software bill of materials per artifact at build time and store it with the artifact.

| Choice | Options | Notes | |---|---|---| | Format | SPDX, CycloneDX | Both are fine; pick one per org and be consistent; verify current spec versions upstream | | Generation | syft, Trivy, cdxgen, buildkit plugins, CI-native | Must include transitive dependencies and versions | | Storage | Registry attestation, artifact store, SBOM database | Keyed by image digest, not release name | | Use | Vulnerability impact, license compliance, incident response, customer due diligence | An unused SBOM is compliance theater |

Quality checks: the SBOM should enumerate OS packages, language packages, and their versions, with package identifiers resolvable to advisories. Validate at build time; reject empty or truncated documents. Regenerate on every build — a rebuild with a new base or dependency produces a new SBOM.

VEX (Vulnerability Exploitability eXchange) lets you publish "not affected" determinations with reasoning so scanners, customers, and auditors stop re-reporting accepted findings. Publish VEX for every accepted risk with a rationale.

SLSA and provenance

SLSA (Supply-chain Levels for Software Artifacts) is a vocabulary for build integrity. The current spec uses tracks and levels; verify the current structure upstream rather than memorizing old level numbers.

Progression that works in practice:

1. Build definition in Git. One versioned definition per artifact; the pipeline is code and reviewed.
2. Provenance generated. CI or a dedicated builder emits an in-toto-style attestation describing source, builder, and parameters; store it with the artifact.
3. Hardened builds. Ephemeral, isolated build environments; the build cannot forge its own provenance; signing keys are held outside the build step.
4. Distribution integrity. Artifacts and provenance are signed; consumers verify before deploy.

Implementation order:

• Digest-addressed artifacts and immutable registries so tags cannot be re-pointed.
• Provenance available and signed for every release.
• Verification in admission and, where feasible, at runtime.
• Reproducible builds where the ecosystem supports it; treat as an enhancement, not a prerequisite.

Sigstore, cosign, and verification

• Keyless signing: the CI workload's OIDC identity is exchanged for a short-lived certificate (Fulcio), and the signature plus certificate are recorded in a transparency log (Rekor). No long-lived private key to steal.
• Key-based signing: use KMS-backed keys where keyless is not permitted; file-based keys with passphrases are the weakest option.
• Sign the digest, then attach SBOM and provenance as attestations to the same digest.
• Verification must bind identity. Checking a signature without checking who signed it is security theater.
• Store signatures and attestations as OCI artifacts in the registry so admission does not depend on outbound transparency-log access; mirror the log entries you rely on.
• Plan for key/certificate rotation and log availability; verification outages block deploys.

# Sign an image keyless in CI (uses the workflow OIDC identity)
cosign sign --yes ghcr.io/example/api@sha256:REPLACE_DIGEST

# Verify with identity binding
cosign verify ghcr.io/example/api@sha256:REPLACE_DIGEST \
  --certificate-identity "https://github.com/example/api/.github/workflows/release.yml@refs/heads/main" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com"

# Attach an SBOM attestation
cosign attest --yes --predicate sbom.spdx.json --type spdxjson ghcr.io/example/api@sha256:REPLACE_DIGEST

Admission verification options: Kyverno verifyImages, Sigstore policy-controller, Ratify. Keep verification webhooks highly available and scope rules narrowly. See ./05-container-cloud-security.md.

Dependency provenance

| Ecosystem | Provenance mechanism (verify current status upstream) | |---|---| | npm | Registry provenance attestations from supported CI providers; lockfile integrity hashes | | PyPI | Trusted Publishing with attestations for supported build backends | | Go modules | go.sum checksums, module proxy and checksum database | | Maven | Central publishing requirements and signature/checksum validation | | Containers | cosign signatures and attestations per digest |

Practices:

• Commit lockfiles and enforce them in CI (npm ci, pip install --require-hashes, cargo --locked, go mod verify).
• Review dependency changes: a new transitive dependency is a new trust relationship. Use automated dependency review on PRs, not only post-merge scans.
• Block typosquats and dependency confusion: scope private packages under a reserved scope, use allowlisted registries, configure the package manager to prefer the private registry for your scopes, and check package names against existing ones.
• Prefer dependencies with maintainer diversity, active releases, and signed/provenance-attested publishes; a single-maintainer package with install scripts is elevated risk.
• Disable install scripts where possible (npm ci --ignore-scripts, no post-install network).
• Treat AI-generated or vendored code as third-party code: provenance, review, license, and vulnerability hygiene.

Build hardening

| Control | Implementation | |---|---| | Ephemeral runners | Fresh, short-lived build environments per job; no shared persistent state | | Cloud access | OIDC federation from CI to a narrowly scoped role; no stored cloud keys | | Workflow permissions | Minimal token permissions per job; separate jobs for build, sign, and deploy | | Pinned actions | Third-party CI actions pinned to commit SHAs, not floating tags | | Isolated signing | Signing step runs after build with a key/identity the build cannot reach | | Hermetic builds | Network-restricted during build where the ecosystem allows; vendor dependencies | | Secret access | Build jobs get no production secrets; deploy jobs get no write access to source | | Cache integrity | Key caches by content and treat cache poisoning as a build threat | | Artifact promotion | Build once, promote the same digest across environments; never rebuild per environment | | Logging | Build logs are immutable and retained; record source commit and artifact digest |

Caveat: "build once" conflicts with per-environment rebuilds for reproducibility in some ecosystems; resolve it explicitly — usually by building one artifact and injecting environment config at deploy.

Vendor and third-party risk

• Maintain an inventory of vendors and open-source components with data access and criticality.
• Due diligence for critical vendors: security program, incident notification terms, subprocessor list, data residency, breach history, certification (SOC 2, ISO 27001) with report review, not just the logo.
• Contractual controls: breach notification window, right to audit, data deletion, SLA, and security requirements.
• Monitor vendor advisories and CVEs affecting components you use; map them to your inventory.
• Regulatory drivers to track: the EU Cyber Resilience Act (vulnerability and incident reporting obligations phase in from late 2026, full obligations later — verify dates upstream) and sector rules such as DORA for financial entities, which impose incident reporting clocks. These raise the bar for SBOM, CVD, and vendor notices.
• Open-source consumption is not free: budget for maintaining, patching, or replacing critical dependencies you cannot influence.

Incident-ready supply chain

Prepare to answer these questions in minutes, not days:

• Which deployed artifacts contain component X at version Y?
• Which artifacts were built by a compromised builder or signed by a compromised identity?
• Which systems consume the affected artifact, and what is the rollback target?
• Can we revoke a signing identity and re-verify the fleet?

If answering requires an email thread, the supply chain program is incomplete.

Anti-patterns

• Signing images but not binding identity in verification.
• SBOM generated at release and never regenerated; base image age untracked.
• Scanning only at build time with no rebuild-and-redeploy path.
• Deploying mutable tags while claiming digest immutability.
• One shared signing key for all repositories, stored as a CI secret.
• CI actions referenced by tag, not SHA.
• Private packages mixed into public scopes without registry pinning.
• Vendor risk assessed once at purchase and never revisited.
• Build jobs with production credentials "because it is easier".
• Assuming the package registry is trustworthy because it is popular.

Checklist

• [ ] Build definitions in Git; runners ephemeral; CI to cloud via OIDC only.
• [ ] CI jobs have minimal permissions; third-party actions pinned to SHAs.
• [ ] Artifacts built once, digest-addressed, and promoted across environments.
• [ ] Provenance generated and signed for every artifact.
• [ ] Images and attestations signed (keyless or KMS); verification binds identity.
• [ ] Registry immutability enabled; admission verifies signatures and attestations.
• [ ] SBOM produced, validated, stored, and signed per digest; VEX for accepted risks.
• [ ] Lockfiles enforced; dependency review on PRs; install scripts disabled where possible.
• [ ] Continuous scanning of running artifacts; rebuild cadence defined and measured.
• [ ] Vendor inventory and due diligence for critical vendors; incident notification terms.
• [ ] Impact queries ("who ships component X") answerable from an SBOM-linked inventory.

Cryptography

Scope: applied crypto engineering — AEAD, password hashing, TLS configuration, KMS/HSM key management, and post-quantum migration.

Rules of engagement

1. Use vetted, high-level libraries (libsodium, Tink, Go crypto, platform crypto). Do not implement primitives or modes.
2. Prefer misuse-resistant APIs: the library should handle nonces, padding, and key separation for you.
3. Never invent a scheme, protocol, or custom "encryption" from a hash.
4. Separate keys by purpose (encryption, signing, key wrapping, tokens); never reuse one key for two operations.
5. Authenticate ciphertext (AEAD or encrypt-then-MAC); never use raw CBC/CTR without integrity.
6. Generate randomness only from the platform CSPRNG; never seed your own.
7. Plan for rotation and crypto agility from the first version; store algorithm/key identifiers with ciphertext.
8. Compliance floors: use algorithm suites allowed by current standards (NIST, BSI, ANSSI); FIPS 140-3 validation when mandated, and verify current validation status upstream.

AEAD

| Algorithm | Nonce size | Use when | Notes | |---|---|---|---| | AES-256-GCM | 96 bits recommended | Hardware AES available; general default | Random nonces safe up to ~2^32 messages per key; nonce reuse is catastrophic | | ChaCha20-Poly1305 | 96 bits | No AES acceleration, mobile, software paths | RFC 8439; same nonce-reuse discipline | | AES-GCM-SIV | 96 bits | Nonce-misuse tolerance desired | RFC 8452; costs throughput | | XChaCha20-Poly1305 | 192 bits | Large random nonce needed | libsodium ecosystem |

Nonce management (the dominant failure mode):

• Prefer a counter or deterministic scheme with persistent state per key; never reuse a nonce with a key.
• With random nonces, rotate the key long before the birthday bound: cap messages per key (for example 2^32 for 96-bit random, lower for high assurance) and document the cap.
• Bind context as associated data: version, key id, tenant, purpose, record id. AAD prevents cross-context ciphertext substitution.
• Store key_id, algorithm, and nonce with the ciphertext; decryption fetches the right key and verifies integrity before use.
• Consider key commitment when ciphertexts from different keys must be distinguishable; standard GCM does not provide it.

AEAD does not solve: replay (add timestamps/sequence numbers), key management (see below), or access control.

Hashing and key derivation

| Purpose | Primitive | Notes | |---|---|---| | Integrity / content id | SHA-256, SHA-512, SHA-3 | Collision resistance; not for passwords | | Message authentication | HMAC-SHA-256 | Constant-time verify; separate keys per context | | Key derivation from high-entropy secret | HKDF (SHA-256+) | Salt plus context info; never skip the expand step | | Key derivation from password | Argon2id, scrypt, bcrypt | Memory-hard, tuned parameters (below) | | Password verification | Argon2id | Preferred; upgrade path for legacy hashes |

Password hashing

Argon2id is the default. Parameters trade memory, time, and parallelism; OWASP-style floors (verify current guidance upstream):

| Profile | Memory | Iterations | Parallelism | |---|---|---|---| | Minimum interactive | 19 MiB | 2 | 1 | | Recommended interactive | 46-64 MiB | 1-2 | 1 | | Higher assurance | 64-128 MiB | 3+ | 1-4 |

Tuning rule: target roughly 50-100 ms per verification on production hardware, use as much memory as the deployment can afford, and store parameters in the hash string so they can be raised over time. Rehash on successful login when parameters increase.

Legacy migration: bcrypt (work factor 10-12 minimum, 72-byte input limit), scrypt (N=2^17, r=8, p=1 as a starting point), PBKDF2 only where required by legacy/compliance (high iteration count). Verify a legacy hash, then rehash with Argon2id on next login.

Never: unsalted hashes, single-round SHA-256 passwords, "pepper" stored next to the hash, client-side-only hashing that becomes the verifier.

TLS configuration

| Setting | Current practice | |---|---| | Versions | TLS 1.3 preferred; TLS 1.2 acceptable with restricted suites; TLS 1.0/1.1 disabled | | Cipher suites | AEAD only: TLS 1.3 defaults; TLS 1.2 with ECDHE + AES-GCM/ChaCha20 | | Key exchange | ECDHE with strong groups; hybrid post-quantum where supported (below) | | Certificates | ECDSA P-256 or RSA-2048+; automate issuance and renewal (ACME) | | Certificate lifetime | Shrinking: CA/Browser Forum ballots step toward short lifetimes (200 days from 2026, then 100, then ~47 by 2029 — verify dates upstream). Automate. | | HSTS | Enabled with a long max-age after verifying subdomains; preload only when committed | | OCSP/CRL | Prefer OCSP stapling; do not rely on client-side soft-fail revocation | | 0-RTT | Disable for non-idempotent requests; replay risk | | Renegotiation/compression | Disabled | | mTLS | Use for service identity; validate client certs against a private CA and check identity, not just possession | | Pinning | Avoid in browsers; use for controlled service-to-service only with a rotation plan |

ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305;
ssl_session_tickets off;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

Internal traffic is not exempt: encrypt service-to-service (mesh mTLS or workload-level TLS) and databases in transit. Certificate management is an availability dependency — monitor expiry and automate renewal with tested rollback.

Key management

Lifecycle for every key: generate in a managed boundary, distribute controlled, use, rotate, revoke, destroy. If any step is missing, the key is a liability.

| Layer | Function | Storage | Rotation | |---|---|---|---| | Root/CMK | Wraps KEKs; trust anchor | HSM/KMS, never exported | Rare, dual control, ceremony | | KEK | Wraps DEKs; may be per-tenant | KMS/HSM | Scheduled and on compromise | | DEK | Encrypts data/messages | Envelope-encrypted, or memory only | Per object/message or short period | | Signing keys | Signatures and tokens | KMS/HSM, non-exportable | Scheduled; support key rollover and old-key verification | | Secrets | Passwords, API keys | Secret manager | Automated, short-lived |

Practices:

• Envelope encryption with per-tenant KEKs for isolation; track KMS request quotas because throttling is an outage.
• Never log key material, never put keys in code, config, images, or CI variables.
• Rotation must be online: write with the new key while reading old keys; rewrap in the background; verify with a drill.
• Revocation path: know how to invalidate a key and its derived artifacts (tokens, signatures) within the incident-response time budget.
• Separation of duties: key administrators manage keys, not data; use quorum for destructive operations; log every key use.
• HSM/FIPS: choose FIPS 140-3 validated modules when required; verify the module and the exact operation is in scope for the validation.
• Backups: encrypted key backups with split custody; test restore. A lost root key is an unrecoverable data loss, and an unguarded backup is a breach.

Post-quantum transition

Standards: NIST FIPS 203 (ML-KEM, key encapsulation), FIPS 204 (ML-DSA, signatures), FIPS 205 (SLH-DSA, hash-based signatures), finalized in 2024; additional signature schemes are in progress. Verify current status upstream. Guidance and timelines: NIST IR 8547 (deprecate classical public-key crypto in the 2030s) and CNSA 2.0 for national-security systems; sector regulators publish their own deadlines.

Why act now: harvest-now-decrypt-later makes long-lived confidential data at risk today even before quantum computers exist. Signatures are less urgent but need crypto agility to migrate.

Migration roadmap:

1. Inventory. Find every use of RSA/ECDH/ECDSA: TLS endpoints, SSH, code signing, document signing, VPN, KMS key types, embedded devices, and long-lived data at rest.
2. Prioritize confidentiality first. Encrypt data that must stay secret for 10+ years with hybrid key establishment now where supported.
3. Hybrid, not pure. In TLS, negotiate hybrid key exchange (for example X25519+ML-KEM-768, deployed across major browsers and libraries; verify current support upstream). Hybrid preserves classical security if the new scheme is broken.
4. Build agility. Version algorithms and keys in protocols and stored data; avoid hard-coding key sizes in serialization formats; make "swap the algorithm" a config change, not a rewrite.
5. Signatures second. Plan ML-DSA/SLH-DSA rollouts for code signing, firmware, and long-lived signed artifacts; hybrid certificates and dual signatures ease the transition.
6. Test. Run interop and performance tests; PQC keys and signatures are larger and slower in some paths; measure before committing.
7. Govern. Track standards, regulator deadlines, and vendor support; record decisions in ADRs.

Do not use unvetted algorithms, do not deploy pure PQC in a single leap for critical paths without interoperability testing, and do not let long-lived data stay on classical-only key exchange once hybrid is available.

Anti-patterns

• Home-grown crypto, custom modes, or "encrypt with SHA-256".
• AES-CBC without integrity, ECB mode, fixed IVs, nonce reuse.
• Keys in source, environment dumps, Terraform state, or CI variables.
• One key for all tenants and all purposes with no rotation plan.
• Password hashing tuned for speed; unsalted hashes; MD5/SHA-1 anywhere.
• TLS terminated with verify=false, expired certs ignored, or mixed HTTP fallbacks.
• Storing ciphertext without key id or algorithm, making rotation impossible.
• Signature verification that ignores the signer identity or uses alg from the token.
• Treating PQC as a 2035 problem; skipping hybrid key exchange on long-lived data.
• Backup keys without custody or restore drills.

Checklist

• [ ] AEAD used everywhere; nonce discipline documented with per-key message caps.
• [ ] Context bound as associated data; ciphertext stores key id, algorithm, and nonce.
• [ ] Passwords hashed with Argon2id using tuned parameters and a legacy upgrade path.
• [ ] CSPRNG used for all keys, nonces, tokens, and identifiers.
• [ ] TLS 1.2+ restricted to AEAD and PFS; 1.0/1.1 disabled; 0-RTT controlled.
• [ ] Certificate issuance and renewal automated; expiry monitored; HSTS configured.
• [ ] Key hierarchy documented with purpose, storage, rotation, revocation, and owner.
• [ ] Envelope encryption per tenant where isolation matters; KMS quotas monitored.
• [ ] Rotation and revocation drills executed; old keys usable for reads during rollover.
• [ ] Crypto inventory includes PQC exposure; hybrid key exchange planned or deployed.
• [ ] Algorithm and key identifiers versioned for agility.

LLM Security

Scope: securing LLM, RAG, and agent features — prompt injection, tool and agent abuse, data exfiltration, guardrails, red teaming, and evaluation.

Why LLM security is different

Three properties break classical assumptions:

1. Instruction and data share a channel. Model input mixes system instructions, user input, retrieved content, and tool output into one token stream. There is no reliable parser that distinguishes "instructions" from "data", so injection is a design condition, not a bug to patch once.
2. Behavior is probabilistic. The same input can take different paths. Controls must be deterministic around the model, not inside it.
3. The model is not a trust boundary. A model asked to enforce policy will sometimes comply with an attacker. Authorization, validation, and egress control must live in ordinary code.

Design rule: treat the model as an untrusted, creative component inside a system whose other parts enforce security.

Prompt injection

Direct injection: the user supplies "ignore previous instructions..." and the model complies. Severity depends on what the model can reach.

Indirect injection: malicious instructions arrive in content the model processes — a retrieved document, web page, email, PDF, ticket, code comment, commit message, calendar invite, or tool result. This is the dominant risk because the attacker does not need access to the user session.

Why prompt-based defenses fail alone: any filter expressed in natural language can be overridden by natural language, and classifiers produce false negatives under novel phrasing and encoding (base64, homoglyphs, other languages).

Defense in depth, ordered by leverage:

| Control | Effect | Limits | |---|---|---| | Least privilege for tools and data | Caps the blast radius of any successful injection | Requires real authz, not model-side checks | | Human approval for irreversible actions | Blocks damage at the last mile | Adds friction; reserve for high-impact | | Tool parameter validation | Rejects off-policy operations in code | Only as good as the policy and schemas | | Egress allowlists and DLP | Stops exfiltration channels | Must cover all output paths | | Spotlighting / delimitering | Marks untrusted content as data | Heuristic; attacker can attempt to escape | | Instruction hierarchy | System > developer > user precedence at training time | Reduces, never eliminates, injection | | Input/output classifiers | Catch known patterns | Cheap layer only; never the primary control | | Dual-LLM / quarantined processing | A privileged model never sees raw untrusted text | Complexity and latency; strong containment |

Practical architecture for assistants with tools:

1. Separate the planner (sees trusted instructions) from the reader (processes untrusted content) and pass only structured, validated data between them.
2. Never let a model's output directly become a shell command, SQL statement, URL fetch, or file path; route through typed tool interfaces with allowlists.
3. Keep secrets out of prompts and context entirely — assume the system prompt is public (see LLM07 system prompt leakage).
4. Log the full tool-call chain with inputs and decisions; injection investigations need the provenance of every instruction.

Tool and agent abuse

Autonomous agents combine injection with real authority, so the controls are architectural.

| Risk | Example | Control | |---|---|---| | Excessive agency | Agent can delete resources or send email without approval | Minimal tool set, read-only default, approval for writes | | Confused deputy | Agent uses its own broad credentials on behalf of a user | Per-user delegated tokens; never a shared super-identity | | Tool description injection | Malicious tool metadata manipulates the planner | Treat tool metadata as untrusted; vet tool sources; pin versions | | Parameter injection | Tool arguments contain attacker-controlled paths/URLs | Schema validation plus semantic allowlists in the tool | | Credential passthrough | Agent forwards its token to downstream systems | Token exchange with audience and scopes per call | | Runaway loops / cost | Agent retries forever or spawns sub-agents | Step, time, and cost budgets; circuit breakers | | Unsafe code execution | Generated code runs with network and credentials | Sandbox with no secrets, bounded CPU/memory, egress allowlist | | MCP and plugin supply chain | Third-party tool servers with broad scopes | Pin servers, review code, scope tokens per server, no token passthrough |

For MCP-style tool ecosystems specifically: authenticate the server, authorize the client, bind tokens to a single server (audience restriction), avoid passing through the user's upstream token, and treat every tool description and result as untrusted input. Verify the current protocol security guidance upstream.

Approval design: require human confirmation for actions that are irreversible, externally visible, financial, or that grant access. Show a faithful representation of the action (recipient, amount, data leaving the boundary), not a model-generated summary.

Data exfiltration and privacy

Exfiltration paths to close:

• Markdown images and links that beacon data to attacker URLs; strip or proxy image URLs in rendered output.
• Tool calls that fetch attacker-controlled URLs (SSRF and DNS exfil); apply egress allowlists.
• Output encoded in base64, steganographic whitespace, or tokens that a downstream client decodes.
• RAG retrieval crossing tenant boundaries; authorization must be enforced at query time per user, not by pre-filtering an index globally.
• Embedding inversion: vectors can leak information about their source text; protect the vector store like the source data.
• Logs and traces containing prompts and tool payloads with PII or secrets; apply retention and redaction.

Controls: tenant-isolated indexes with per-request authz filters, DLP on outputs and egress, output encoding for the rendering context, prompt/response logging with redaction and access control, and data minimization in context assembly.

Privacy and compliance: map data flows to the threat model (./01-threat-modeling.md), avoid sending regulated data to providers without contractual coverage, keep data residency in mind, and record model/provider inventory for AI governance (NIST AI RMF, EU AI Act obligations phase in over 2025-2027 — verify dates upstream).

Guardrails

Layer them; no single guardrail is sufficient, and weak guardrails create false confidence.

| Layer | Examples | Failure mode to avoid | |---|---|---| | Input filtering | Injection detectors, length/size limits, allowed content types | Treating classifier output as authorization | | Context assembly | Spotlighting, provenance labels, strict templates, no secrets in prompts | Assuming delimiters are unforgeable | | Tool mediation | Typed schemas, allowlists, parameter validation, per-tool scopes | Model-side "check before calling" logic | | Action approval | Human-in-the-loop for writes and high-impact actions | Approval fatigue from over-prompting | | Output validation | Schema checks, banned-content filters, PII redaction | Rendering raw markdown with URLs | | Egress control | Destination allowlists, DLP, no direct internet from sandboxes | Blocking only the obvious channel | | Monitoring | Anomaly detection on tool calls, volume, and data movement | Logging prompts without alerting |

Guardrail engineering: version prompts and policies like code, test them against adversarial suites, measure false positive and false negative rates, and never deploy a guardrail that blocks a critical path without a documented override.

Provider and deployment considerations

• Know your data path: managed API, private endpoint, self-hosted, or on-device. Each changes the trust boundary, the data-processing agreement, and the incident path.
• Contract for security: no training on your data where required, retention windows, regional processing, subprocessors, and breach notification terms. Verify current provider terms upstream.
• Keep model and provider inventory with versions, approval status, data classes allowed, and owner; this is the artifact auditors and incident responders ask for.
• Treat model files and weights as supply-chain artifacts: download from trusted sources, verify hashes or signatures, and store in a controlled registry.
• Control fallbacks: if a provider is unavailable, the failover path must not silently send data to an unapproved endpoint.
• Pin model versions where the provider allows it; an unpinned model can change behavior overnight and invalidate your evaluations.

Red teaming and evaluation

Adversarial testing is continuous, not a one-time launch gate. Model updates, prompt changes, tool changes, and retrieval corpus changes all invalidate prior assurance.

Tooling categories (verify current capabilities upstream): garak for broad probing, PyRIT-style orchestration for automated multi-turn attacks, promptfoo-style harnesses for regression suites, and custom suites built from your own threat model.

Test dimensions:

| Dimension | Example probes | Success metric | |---|---|---| | Direct injection | Role-play and override attempts | Policy violation rate | | Indirect injection | Poisoned document, email, or web page in the pipeline | Action taken on attacker instruction | | Exfiltration | "Summarize and send to this URL" via tool or markdown | Data reaching an unapproved destination | | Tool misuse | Off-scope file paths, commands, API calls | Out-of-policy tool invocations | | System prompt extraction | Requests to reveal instructions | Secret or policy leakage | | RAG isolation | Queries across tenants, permission-elevating prompts | Cross-tenant retrieval | | Availability/cost | Token bombs, recursive loops | Cost or latency beyond budget | | Over-refusal | Benign edge-case prompts | Task success rate on legitimate requests |

Evaluation practice: keep a versioned suite with expected outcomes, run it on every change in CI where feasible, track rates over time rather than pass/fail, and include a red-team review for new tool grants. Treat "injection success rate > 0" as expected; the question is whether the blast radius is acceptable.

Incident response for AI features

• Add model, prompt version, tool grants, retrieval corpus version, and provider to the incident timeline.
• Ability to disable a tool or model route quickly is the equivalent of a kill switch: design it before launch.
• Preserve prompts and tool-call logs for the affected window; they are the forensic record.
• Coordinate provider disclosure when the incident involves the model or platform; see ./09-incident-response.md.

Anti-patterns

• Relying on "ignore malicious instructions" in the system prompt.
• Putting secrets, credentials, or internal policy details in the system prompt.
• Giving an agent broad credentials "to simplify integration".
• Letting model output flow to eval, shell, SQL, or raw HTML without validation.
• Rendering model output as markdown/HTML with unrestricted image and link URLs.
• Enforcing RAG access control in the UI instead of the retrieval query.
• One shared vector index for all tenants.
• Treating a passing red-team run as permanent assurance.
• Logging full prompts containing PII without access control or retention limits.
• Approving tool actions with a model-written summary the human cannot verify.

Checklist

• [ ] Model treated as untrusted; authz, validation, and egress enforced in code.
• [ ] Tools scoped to the minimum, read-only by default, with per-user credentials.
• [ ] Irreversible and externally visible actions require human approval with faithful details.
• [ ] Untrusted content spotlighted and processed in a quarantined path where warranted.
• [ ] No secrets in prompts or context; system prompts assumed public.
• [ ] Output validated by schema and rendered with URL/sanitization controls.
• [ ] Egress allowlists and DLP cover all model and agent channels.
• [ ] RAG enforces per-request, per-tenant authorization at query time.
• [ ] Code execution sandboxed with no secrets, bounded resources, and no default network.
• [ ] Adversarial eval suite versioned and run on model, prompt, tool, and corpus changes.
• [ ] Prompt/tool logs retained with redaction, access control, and defined retention.
• [ ] Kill switch for models, tools, and agent identities tested.
• [ ] AI incident playbook includes model/prompt/corpus provenance.

Incident Response

Scope: security incident handling — detection and triage, containment, forensics, communication, postmortems, tabletops, and metrics.

Lifecycle

Align with NIST SP 800-61r3 (which maps response activities onto the CSF 2.0 functions, Govern, Identify, Protect, Detect, Respond, Recover — verify the current revision upstream) or your regulator's framework. The phases are not strictly sequential; containment and analysis overlap.

1. Prepare — playbooks, access, tooling, contacts, backups, and rehearsals. Everything else is improvisation without this.
2. Detect and analyze — validate the signal, scope it, classify severity, start the record.
3. Contain — stop the spread while preserving evidence; eradicate the root cause.
4. Recover — restore service from known-good state and watch for recurrence.
5. Learn — blameless postmortem, action items, updated playbooks and controls.

Severity classification

Define severity before the incident, not during it. A workable four-level scheme:

| Severity | Definition | Examples | Targets | |---|---|---|---| | SEV1 / critical | Confirmed breach, active data exfiltration, or safety impact; major customer or regulatory exposure | Ransomware, production data theft, compromise of signing keys or cloud root | Immediate page, incident commander and exec notified within minutes, updates every 30-60 min | | SEV2 / high | Confirmed compromise limited in scope, or active exploitation with partial controls holding | Compromised service account, malware on one host, exposed credential with access | Page on-call, updates hourly, containment within hours | | SEV3 / medium | Suspicious activity needing investigation, or a vulnerability with a credible path | Anomalous admin action, unpatched critical on an exposed host | Ticket with an owner, response within a business day | | SEV4 / low | Policy violation, scanner finding without evidence of exploitation | Misconfiguration without exposure, phishing email reported by one user | Normal backlog with SLA |

Decision rule: when unsure between levels, start one higher and downgrade with evidence. Also define severity escalation triggers explicitly: any confirmed data access, any privileged credential compromise, any detection of lateral movement.

Detection and triage

Detection sources: SIEM correlation, EDR/runtime alerts, cloud audit logs, identity provider anomaly detection, WAF and gateway alerts, secret scanning, supply-chain advisories, customer reports, and employee reports. Fill gaps where a critical path has no signal (for example, no audit logging on the data plane).

Triage checklist for every alert:

1. Validate — is this real, and is it happening now? Reproduce the signal, not the assumption.
2. Scope — which identities, hosts, accounts, and data are involved? Pull the last 90 days of relevant history, not the last hour.
3. Determine initial access — how did the actor get in and when? Check identity logs, exposed services, recent changes, and known vulnerable components.
4. Check persistence — new credentials, tokens, scheduled jobs, webhooks, CI secrets, OAuth grants, SSH keys, and cloud roles.
5. Assess data impact — what could have been read or exfiltrated? Classification and volume matter for notification.
6. Classify severity and open the record — single source of truth with timestamps in UTC.
7. Page the right people — incident commander, comms, legal, and domain owners per severity.

Alert quality matters more than alert volume: tune to high-fidelity signals, enrich with asset and identity context, and track false-positive rates. A noisy SIEM that everyone ignores is an outage of detection.

Containment

Containment choices trade speed against evidence preservation and availability. Decide with the incident commander, and record the reason for each action.

| Action | Use when | Risk | |---|---|---| | Revoke sessions and rotate credentials | Credential compromise suspected | Attacker may notice; legitimate users disrupted | | Disable an identity or service account | Actor uses a specific identity | May break automations; preserve logs first | | Isolate a workload/host from the network | Active lateral movement or runtime malware | Memory-resident evidence may be lost; capture before isolating if possible | | Block egress destination or domain | Exfiltration in progress | Attacker rotates infrastructure; keep logging exits | | Roll back to known-good artifact/config | Compromised deploy or dependency | Rollback may reintroduce the vulnerable version; patch forward first | | Disable a feature or tool | Injection or abuse through a specific path | Product impact; needs a re-enable plan | | Rotate keys/signing identities | Key compromise | Wide blast radius; requires a rotation runbook |

Order of operations: preserve the highest-volatility evidence first (memory and running state), then contain. When speed is essential to stop data loss, contain first and accept partial evidence; document the tradeoff.

Containment is not eradication: re-imaging without finding initial access and persistence guarantees a second incident.

Forensics and evidence

• Order of volatility: CPU/registers, memory, network connections, running processes, disk, remote logs, backups.
• Capture memory and disk images with tested tooling before rebooting; hash every image at collection (sha256sum) and record collector, time, and location.
• Cloud forensics: enable and export audit trails (management, data, and identity logs), snapshot disks and memory where supported, and preserve container images by digest. Logs with short retention must be exported early.
• Containers and Kubernetes: capture pod spec, image digest, environment references, service account, and node logs; treat the node as compromised if privileged access occurred.
• Build a timeline in UTC from all sources; keep a chain-of-custody record for anything that may reach legal or regulatory review.
• Engage legal early for privilege and notification duties; do not let the security team silently make legal calls.
• Avoid "cleaning up while investigating": every command changes state. Log your own actions.

Communication

Roles to fill (one person may hold several in small teams): incident commander, deputy/scribe, communications lead, legal/compliance, executive sponsor, and domain leads.

| Audience | Cadence | Content | |---|---|---| | Incident team | Continuous in a dedicated channel | Facts, hypotheses labeled as such, decisions, tasking | | Leadership | Per severity policy (30-60 min for SEV1) | Impact, status, next update time, decisions needed | | Customer-facing teams | Before public statements | Approved language, FAQ, escalation path | | Customers/partners | Per legal and contractual duties | Validated facts, remediation, contacts; no speculation | | Regulators | Regulator-specific clocks | See below | | Employees | After customer notification | What happened, what to say, where to route questions |

Regulatory clocks to know and verify with counsel (all are jurisdiction- and sector-specific; confirm the current requirements upstream):

• GDPR: personal data breach notification to the supervisory authority within 72 hours of awareness where required; data subjects without undue delay when high risk.
• SEC (US public companies): material cybersecurity incident disclosure generally within four business days of determining materiality.
• DORA (EU financial entities): major incident reporting with very short initial notification deadlines (hours), intermediate and final reports following.
• EU Cyber Resilience Act: manufacturers report actively exploited vulnerabilities and severe incidents to the designated CSIRT on short clocks (an early-warning stage around 24 hours and follow-ups within days, phasing in from late 2026 — verify dates upstream).
• Sector rules (health, energy, telecom) add their own clocks; pre-map them per jurisdiction.

Practice the clocks in tabletop exercises; discovering the notification duty during a real incident is too late. Prepare holding statements and notification templates in advance, and route all external communication through legal and comms.

Eradication and recovery

• Identify and remove root cause and every persistence mechanism; assume the actor left more than one.
• Rebuild from trusted sources rather than patching a compromised system: new credentials, re-imaged hosts, redeployed artifacts from verified digests.
• Rotate everything the actor could have touched: passwords, tokens, API keys, certificates, and cloud roles; invalidate sessions.
• Restore data from backups verified clean (attacker may have compromised backups); confirm integrity before returning to service.
• Return to production in stages with heightened monitoring; define explicit exit criteria and a rollback path.
• Watch for re-entry for a defined period after recovery; track it in the incident record.

Postmortem

Blameless and factual. The goal is system change, not blame. Do it within days while memory is fresh.

Structure:

1. Summary, severity, customer and data impact, duration.
2. Timeline in UTC with detection, decision, and action points, including what took so long.
3. Root cause and contributing factors (technical, process, organizational); use causal trees or "five whys" without stopping at "human error".
4. What worked and what did not, including detection quality and tooling gaps.
5. Action items: specific, owned, dated, and tracked in the normal backlog; include control and detection improvements.
6. Appendix: evidence references and regulatory notifications made.

Publication practice: share broadly (redacted where needed). An unpublished postmortem teaches one team; a published one teaches the company.

Tabletop exercises

• Scenario types: ransomware, cloud credential theft, supply-chain compromise, insider data theft, LLM exfiltration, lost device, regulatory notification race.
• Cadence: at least quarterly for the security team, twice yearly for engineering leadership; rotate scenario types.
• Participants: on-call, engineering, comms, legal, and an executive sponsor; include a scribe and an evaluator.
• Inject new facts every 15-20 minutes ("backups are also encrypted", "the regulator just called") to force decisions.
• Test the boring parts: contact tree, out-of-band communication, evidence access, and who can approve emergency changes.
• Output: findings and action items tracked like postmortem actions; re-run scenarios that expose gaps.

Metrics

| Metric | Definition | Use | |---|---|---| | MTTD | Mean time to detect from first activity | Measures detection investment | | MTTA | Mean time to acknowledge | Measures on-call responsiveness | | MTTC | Mean time to contain | Measures playbook effectiveness | | MTTR | Mean time to recover service | Pairs with MTTC; watch for improvement gaming | | Internal detection rate | Incidents found internally vs externally reported | The single best health signal; aim for high internal rate | | Escape rate | Incidents caused by changes that passed review/scanning | Ties response back to prevention | | Action-item completion | Postmortem items closed by the due date | The difference between learning and theater | | Reopen rate | Incidents that recur within a window | Whether root cause was really addressed |

Track trends and outliers, not absolute values, and review metrics with engineering leadership. A metric used for individual performance stops being honest.

Playbook starter list

• Credential compromise (user, service account, cloud key).
• Ransomware/malware on a host or cluster.
• Data exfiltration via application or database.
• Public data exposure (bucket, database, repository).
• Lost or stolen device with corporate access.
• Supply-chain compromise (dependency, CI, artifact).
• LLM/agent abuse or exfiltration (see ./08-llm-security.md).
• Insider misuse and access revocation.
• Denial of service and resource exhaustion.

Each playbook: trigger, severity mapping, first 30 minutes, containment options, evidence to preserve, communication tree, recovery steps, and exit criteria. Test them in tabletops; keep them short enough to follow at 3 a.m.

Anti-patterns

• No written severity definitions; every incident becomes a debate.
• Containment that destroys the evidence needed to find the entry point.
• Fixing the symptom and closing the incident without root cause.
• Postmortems that blame individuals and drive future incidents underground.
• Action items with no owner or date; they never get done.
• Regulatory notification discovered after the deadline.
• Channels outside the incident record (DMs), so the timeline is lost.
• Backups untested against ransomware; restore drills skipped.
• Metrics tracked but never reviewed with engineering leadership.
• Tabletops that never touch communication, legal, or executive decision-making.

Checklist

• [ ] Severity matrix published with escalation triggers and response targets.
• [ ] On-call rota, paging, and out-of-band communication tested.
• [ ] Playbooks for the top scenarios exist, are current, and are exercised.
• [ ] Logging and retention cover identity, control plane, data plane, and network.
• [ ] Evidence collection tooling and custody procedures documented and tested.
• [ ] Legal and regulatory notification matrix maintained per jurisdiction.
• [ ] Communication templates and approval chains pre-drafted.
• [ ] Recovery uses verified backups and trusted rebuilds; exit criteria defined.
• [ ] Blameless postmortems within days; actions owned, dated, and tracked.
• [ ] Tabletop cadence scheduled with executive participation.
• [ ] Detection and response metrics reviewed regularly with leadership.