# FlowMarkup: Engine Implementation Guide

**Version:** 0.9.0
**Date:** 2026-03-28
**Designed by:** Łukasz Nawojczyk
**Copyright:** © 2026 Progralink Łukasz Nawojczyk. All rights reserved.
**License:** Open Web Foundation Agreement 1.0 (OWFa 1.0)

This document specifies requirements for FlowMarkup engine implementations. It was extracted from the main specification ([FLOWMARKUP-SPECIFICATION.md](FLOWMARKUP-SPECIFICATION.md)) to separate the YAML format definition (what valid flows look like) from engine behavior (how engines execute them). Section numbers are preserved for cross-reference continuity.

### Conformance

The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).

### Related Documents

| Document | Purpose |
|---|---|
| [FLOWMARKUP-SPECIFICATION.md](FLOWMARKUP-SPECIFICATION.md) | Format specification (YAML structure, directives, actions, recommendations) |
| [FLOWMARKUP-VALIDATION.md](FLOWMARKUP-VALIDATION.md) | Full catalog of SA rules engines must enforce |
| [FLOWMARKUP-TESTING.md](FLOWMARKUP-TESTING.md) | Testing framework specification (test runner requirements) |
| [FLOWMARKUP-SECRETS.md](FLOWMARKUP-SECRETS.md) | Secrets management and provider SPI |
| [FLOWMARKUP-ERRORS.md](FLOWMARKUP-ERRORS.md) | Error types and ERROR binding |
| [FLOWMARKUP-ENGINE-CROSSLANG.md](FLOWMARKUP-ENGINE-CROSSLANG.md) | Cross-language engine implementation guide |
| JAVA-REFERENCE.md *(planned for a future release)* | Java reference implementations |

---

## 5. Engine Architecture

### 5.1 Action Provider Model

Each action provider:
- Registers a step type name or service name
- Declares a versioned contract (inputs, outputs, errors)
- Ships a JSON Schema file for composition
- Implements execution logic

Provider IDs are reverse-DNS dot-namespaced strings. `progralink.clients.*` is reserved for built-in providers.

### 5.2 Schema Composition

The engine composes the full schema from core + action provider schemas. Each action schema is inserted into `step.oneOf` before the generic `actionStep` catch-all. The action's key is added to the catch-all's `not.anyOf` exclusion list.

```
schema/
  flowmarkup.base.schema.json       <- Core
  flowmarkup.full.schema.json       <- Composed (generated)
  actions/*.action.schema.json      <- Per-action
```

Target published schema filename must be versioned, e.g.: `FlowMarkup-0.9.0.schema.json`.

### 5.3 Static Analysis

**Parse-time desugaring** — before static analysis and execution, the engine MUST apply the following transformation: if `transaction:`, `onRollbackError:`, or `locking:` are present at the `flowmarkup:` root, wrap the `do:` list in an implicit sequential `group:` step carrying those properties. The internal representation becomes identical to an explicit `do: [group: { transaction: ..., onRollbackError: ..., locking: ..., do: [...] }]`. This MUST happen before SA rules run so that SA-ROLLBACK-* and SA-ISO-* apply uniformly to both explicit and implicit transaction groups.

Static analysis catches semantic errors that JSON Schema cannot express. Rules are organized by category with SA-* identifiers:

- **Data flow** -- input completeness, output reachability, type compatibility
- **Control flow** -- loop scope, return/throw coverage, unreachable code
- **Scoping** -- variable names, reserved prefixes, data element immutability (`const:` and `$readonly`), init order
- **Concurrency** -- parallel variable conflicts, handler races, isolation
- **Security** -- secret opacity (SA-SECRET-*), capability grants (SA-RUN-*), CEL introspection (SA-CEL-*)
- **Action-specific** -- SA-EXEC-*, SA-MAIL-*, SA-REQ-*, SA-SVC-*, SA-XML-*, SA-STORAGE-*, SA-SSH-*
- **Streaming** -- SA-YIELD-*, SA-CB-*
- **Expression** -- SA-QUOTE-*, SA-INTERP-*, SA-EXPR-*

The full catalog of rules is in [FLOWMARKUP-VALIDATION.md](FLOWMARKUP-VALIDATION.md).

### 5.4 Security Model

1. **No recursive evaluation** -- expression results are never re-evaluated as expressions.
2. **Resource path validation** -- engine's responsibility (path traversal prevention, allowlists).
3. **Action provider input validation** -- providers MUST sanitize all input parameters. Action providers MUST NOT log resolved secret values.
4. **Resource limits** -- the engine MUST enforce the following limits:
   - **Maximum recursion depth:** default 100 (configurable). MUST NOT exceed 1,000. Exceeding raises `StackOverflowError`.
   - **CEL expression AST depth:** MUST limit to a configurable maximum (default: 32). MUST NOT exceed 128. Exceeding raises `ValidationError` at load time.
   - **CEL collection operation size:** MUST limit maximum collection size for CEL operations (default: 10,000 elements). MUST NOT exceed 1,000,000. Exceeding raises `ResourceExhaustedError`.
   - **Per-instance memory:** MUST enforce a configurable per-flow-instance memory limit. Exceeding raises `ResourceExhaustedError`.
   - **HTTP response size:** MUST enforce a configurable maximum response body size for `request` actions (default: 10 MB). MUST NOT exceed 100 MB. Exceeding raises `ResourceExhaustedError`. The limit applies to the decompressed response body.
   CEL expressions — including inline service calls (`SERVICES.<alias>.<operation>(...)`) — are bounded by the step-level `timeout:`, group-level `timeout:`, and flow-level `timeout:`. A dedicated per-expression timeout is not required; the flow and group timeout hierarchy provides sufficient coverage.
   - **Regex evaluation:** The engine MUST guarantee that `s.matches(regex)`, `$format` regex validation, and any other regex evaluation complete in time linear in the input length, OR enforce a configurable per-evaluation timeout (default: 100ms, maximum: 5s). Engines using backtracking regex engines (Java `Pattern`, Python `re`, JavaScript `RegExp`) MUST enforce the timeout. Engines using linear-time engines (RE2, Rust `regex`) satisfy the linear-time requirement by construction.
5. **CEL sandboxing** -- non-Turing complete, no I/O, no side effects. Host-language introspection MUST be blocked.
6. **CEL extension function restrictions** -- Custom CEL extension functions MUST NOT accept `SecretValue` parameters. SA-SECRET-22 (ERROR) enforces this at static analysis time. Extension function implementations MUST be reviewed for secret leakage before registration.
7. **Environment variable redaction** -- engine MUST redact all `ENV.*` values from logs and traces. SA-ENV-1 (ERROR) rejects flows where `ENV.*` references match common credential patterns (`*_KEY`, `*_SECRET`, `*_TOKEN`, `*_PASSWORD`, `*_CREDENTIAL`, `*_CREDENTIALS`, `*_API_KEY`, `*_APIKEY`, `*_AUTH`, `*_PRIVATE_KEY`, `*_CONN_STRING`, `*_CONNECTION_STRING`, `*_DSN`, `*_PASSPHRASE`, `*_PIN`) and requires use of `SECRET.*` instead. The engine MUST support configurable custom credential patterns at the engine level, so operators can extend SA-ENV-1 detection beyond the built-in list. SA-ENV-2 (ERROR) rejects `set:` assignments where the value is an `ENV.*` reference matching a credential pattern — preventing credential values from being copied into loggable flow variables. SA-ENV-3 (INFO) warns on any `ENV.*` usage, noting that ENV values are not redacted at the application level and recommending `SECRET.*` for any values that may contain credentials. **`ENV.*` provides NO security guarantees for credential material** — no opacity, no mandatory redaction at the application level, no taint tracking. `ENV.*` values are plain strings that can be logged, returned, emitted, and interpolated freely. For any value that may contain credentials, use `SECRET.*` instead.
8. **Global variable tenant isolation (MUST)** -- `GLOBAL.*` namespace MUST be scoped per-tenant. The engine MUST partition the global store by tenant identity such that flow instances belonging to one tenant cannot read, write, list, or observe the existence of `GLOBAL.*` keys belonging to another tenant. The same isolation MUST apply to GLOBAL-scope events (`emit`/`waitFor` with `scope: GLOBAL`), GLOBAL-scope locks (`lock` with `scope: GLOBAL`), and GLOBAL-scope circuit breakers. `has(GLOBAL.key)` from another tenant MUST return `false` using constant-time evaluation (consistent with `has(SECRET.*)` in item 55). Cross-tenant `GLOBAL.*` read attempts MUST be logged as audit events with severity WARN, including the requesting tenant identity, target key name, and timestamp. Engine implementations MUST include conformance tests that verify: (a) cross-tenant `GLOBAL.*` read returns `null`/undefined (not an error revealing key existence); (b) cross-tenant GLOBAL-scope events are never delivered; (c) cross-tenant GLOBAL-scope locks do not conflict; (d) cross-tenant `has(GLOBAL.key)` returns `false` in constant time.
9. **Runtime information exposure** -- `RUNTIME.*` is deny-by-default. Engine MUST NOT include `RUNTIME.*` values in user-visible error messages, error responses, or client-facing API output. Authenticated diagnostic/debug endpoints are exempt.
10. **Capability security** -- monotonic decrease down the call chain. Every flow MUST declare `requires:`. Effective capabilities = `requires(sub-flow) ∩ capabilities(caller) ∩ cap(run-step)`. No implicit inheritance, no same-origin/cross-origin distinction.
11. **`exec` capability** -- deny-by-default. No shell interpretation. Engine MUST validate against allowlist.
12. **Proxy configuration** -- engine-internal. Flows SHOULD NOT read or log proxy environment variables.
13. **`mail` capability** -- deny-by-default. Engine SMTP config is opaque. Engine SHOULD validate addresses and enforce size limits.
14. **Secrets management** -- opaque `SecretValue` handles. Engine MUST redact from all output. Deny-by-default capability. Errors: `SecretAccessError`, `SecretTypeError` (all non-retryable). Secrets come from the engine's secrets provider only — inline secret definitions in `cap:` are not supported (SA-SECRET-19, ERROR).
15. **Action provider secret isolation** -- Action providers MUST NOT include resolved `SecretValue` content in error responses, exception messages, or diagnostic output. The provider SPI contract MUST specify this as a testable requirement. Every engine MUST ship a provider conformance test suite that verifies no resolved `SecretValue` appears in error responses, diagnostic output, or logged messages. Engine-internal debug logs MUST apply the same redaction rules as FLOWMARKUP-SECRETS.md §7.1 audit events. See FLOWMARKUP-SECRETS.md §7.1 for the complete redaction specification.
16. **Redirect origin enforcement** -- when the effective `REQUEST` capability is not `INHERIT`, the engine MUST validate each HTTP redirect target against the effective REQUEST allowlist before following. Redirects to origins not in the allowlist MUST be rejected with `ConnectionError`. This prevents SSRF attacks where an allowed origin redirects to internal or unauthorized endpoints. The engine MUST resolve DNS at connection time and pin the resolved IP for the duration of the request (DNS pinning). The engine MUST validate the resolved IP address — not just the hostname — against private/internal IP ranges (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `127.0.0.0/8`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fd00::/8`, `fe80::/10`) and cloud provider metadata endpoints (`169.254.169.254/32` — AWS/GCP/Azure instance metadata, `fd00:ec2::254` — AWS IMDSv2 IPv6, `metadata.google.internal`) before connecting. This prevents DNS rebinding attacks where an allowed hostname resolves to an internal IP after the initial allowlist check. The engine MUST apply DNS resolution and private/internal IP validation to each redirect target before following. Relative redirects (scheme-relative `//host/path` or path-relative `/path`) MUST be resolved against the original request URL and then validated. Maximum redirect chain depth: 10 (configurable). Each redirect hop counts toward the step's `timeout:`.
17. **Mail sender validation** -- engine MUST restrict `from:` to addresses authorized by SMTP configuration (SPF/DKIM/DMARC alignment). Unvalidated sender addresses risk spoofing and deliverability failures. SA-MAIL-13 (WARN) flags explicit `from:` fields; SA-MAIL-17 (ERROR) flags user-derived `from:` addresses.
18. **Mandatory capability declaration** -- every flow MUST declare `requires:`. `requires: {}` means no capabilities needed (minimal sandbox). The engine MUST reject flows without `requires:` at load time (SA-FLOW-4, ERROR). The engine MUST support integrity-based flow verification (`integrity:` on `run:` steps) and SHOULD support flow signing for production deployments.
19. **TLS certificate verification** -- the engine MUST always verify TLS certificates for all outbound connections (HTTP requests, SMTP). There is no `verifyTLS` option — TLS verification cannot be disabled per-flow. The engine MUST use the operating system's native certificate store (e.g., Windows Certificate Store, macOS Keychain, Linux `/etc/ssl/certs/`) rather than a bundled certificate store embedded in the runtime. This ensures certificate trust follows the platform's security policy and receives OS-level updates. Custom CA certificates for internal services MUST be installed in the OS certificate store or configured at the engine level, not in flow YAML.
20. **Async sub-flow security error escalation (MUST)** -- security errors (`MissingCapabilityError`, `SecretAccessError`, `AuthenticationError`, `AccessDeniedError`) from `async: true` sub-flows are NOT propagated to the parent flow's `catch:` handlers — the parent flow continues unaware. These errors MUST be emitted as mandatory audit events regardless of the global audit logging toggle. These events are unconditional — they MUST be logged even when audit logging is globally disabled (see §5.7). The engine MUST expose a per-flow-instance security error counter via the management API. The engine SHOULD support configurable alerting hooks (e.g., webhook, metric increment) triggered by async security errors.
21. **DNS pinning for remote flow fetches** -- the engine MUST apply the same DNS pinning and private/internal IP validation described in item 16 to all remote flow fetches (HTTP/S and git-backed URLs). The engine MUST resolve DNS at fetch time and validate the resolved IP address against private/internal IP ranges before connecting. This prevents DNS rebinding attacks.
22. **`request` capability** -- deny-by-default. A flow using `request` steps MUST declare `requires: { REQUEST: [...] }` with explicit origin patterns. The engine MUST reject flows with `request` steps that lack a `REQUEST` capability declaration (SA-REQ-12, ERROR).
23. **Resource exhaustion prevention** -- the engine MUST enforce a configurable per-tenant rate limit on SECRET resolution (default: 100 resolutions per second per tenant). The engine MUST return `SecretRateLimitError` (non-retryable) when the limit is exceeded. The engine MUST also enforce per-tenant rate limits on GLOBAL writes (default: 100 writes per second per tenant) and event emission (default: 1,000 events per second per tenant) to prevent resource exhaustion across multiple flow instances. In addition to the per-tenant rate limit, the engine MUST enforce a per-flow-instance rate limit of 10 secret resolutions per second (configurable). This prevents a single runaway flow from exhausting the secrets backend.
24. **XML Security** -- the engine MUST apply the following protections in ALL XML parsing contexts (`decode(XML)`, `parse(XML)`, `xpath()`, `xpathAll()`, `parseAs: XML` auto-decode on `exec`/`ssh`/`request`/`storage`, and RESOURCES `.xml` loading):
    - **XXE prevention:** MUST disable external entity resolution. Java: `XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES = false`, `SUPPORT_DTD = false`, `XMLConstants.ACCESS_EXTERNAL_DTD = ""`, `XMLConstants.ACCESS_EXTERNAL_SCHEMA = ""`. Python: `defusedxml` library or `lxml` with `resolve_entities=False`, `no_network=True`. Go: `encoding/xml` (no entity expansion by default — verify no custom entity resolver is registered). JavaScript/TypeScript: `libxmljs2` with `noent: false`, `nonet: true`; or DOM parser with entity expansion disabled. C#: `XmlReaderSettings.DtdProcessing = DtdProcessing.Prohibit`, `XmlResolver = null`. Rust: `quick-xml` (no entity expansion by default). Engine implementations MUST include cross-language conformance tests that verify XXE prevention by attempting to load XML documents containing external entity references (file://, http://) and verifying they are rejected. External entities can exfiltrate files, probe internal networks, and execute server-side request forgery.
    - **Billion-laughs / exponential entity expansion:** MUST enforce a configurable entity expansion limit (default: 10,000) and a configurable maximum entity nesting depth (default: 10). Exponential entity expansion can consume unbounded memory and CPU.
    - **Remote file access via external entities:** MUST disable `XInclude` processing (`setXIncludeAware(false)`) and disallow `file://`, `http://`, `https://`, `ftp://` protocol handlers in entity resolution. Entities that reference external resources MUST be rejected, not silently ignored.
    - **Oversized document protection:** MUST enforce a configurable maximum XML document size (default: 10 MB, same as HTTP response size limit). Documents exceeding the limit MUST raise `ResourceExhaustedError` before parsing completes. MUST enforce configurable limits on maximum element depth (default: 100), maximum attribute count per element (default: 100), and maximum element name length (default: 1,000 characters).
25. **Content schema non-validation** -- the engine MUST NOT automatically validate loaded content (XML, JSON, YAML) against schemas declared within the content itself (e.g., XSD/DTD references in XML, `$schema` in JSON, YAML tags). Content schema validation is a potential vector for SSRF (schema URL fetching), denial-of-service (recursive/circular schema references), and arbitrary resource loading. If content schema validation is needed, flow authors use FlowMarkup's `types:` with `$schema` and explicit validation steps.
26. **`STORAGE` capability** -- deny-by-default. A flow using `storage` steps MUST declare `requires: { STORAGE: ... }`. The engine MUST validate `storage.url:` — resolve bare aliases from the engine's or parent-flow's STORAGE mapping, then match the resolved URL against alias or URL-pattern capability declarations before executing. Path prefixes, when declared, MUST be validated at runtime before provider invocation.
27. **Storage path validation** -- the engine MUST validate storage paths to prevent path traversal attacks. The engine MUST apply the following checks to ALL storage paths before passing them to the storage backend:
    - **Traversal sequences:** MUST reject paths containing `..` segments (literal `../`, `..\`, or URL-encoded `%2e%2e%2f`, `%2e%2e%5c`). SA-STORAGE-17 (ERROR) catches URL-encoded traversal at static analysis time; runtime validation catches dynamic values.
    - **Null bytes:** MUST reject paths containing null bytes (`\0`, `%00`). Null bytes can truncate paths in C-based backends, bypassing suffix checks.
    - **Absolute path injection:** MUST reject paths that start with `/` or `\` when a `path:` field is appended to a `url:` base — the combined path MUST stay within the base endpoint's namespace.
    - **Normalization order:** Path validation MUST occur AFTER all decoding (URL decoding, Unicode normalization NFC) and BEFORE passing to the storage backend. Double-encoding attacks (`%252e%252e`) MUST be caught by decoding iteratively until stable or rejecting paths that change after a second decode pass.
    - **Provider-specific:** S3 key validation (maximum 1024 bytes, no control characters). Filesystem path validation (no symlink traversal outside storage root — engine MUST resolve symlinks and verify the canonical path is within the configured root). SFTP/FTP path validation follows filesystem rules.
    - **Windows path handling:** On Windows platforms (or when the storage backend is a Windows filesystem), the engine MUST additionally: (1) reject paths containing reserved device names (`CON`, `PRN`, `AUX`, `NUL`, `COM1`–`COM9`, `LPT1`–`LPT9`) in any path segment, with or without extensions (e.g., `NUL.txt`); (2) reject paths containing characters forbidden by NTFS (`<`, `>`, `:`, `"`, `|`, `?`, `*`, and control characters 0x00–0x1F); (3) reject alternate data stream syntax (`:` after the filename, e.g., `file.txt:hidden`); (4) apply case-insensitive comparison for traversal detection (Windows filesystems are typically case-insensitive); (5) normalise path separators (`\` → `/`) before applying all other validation rules. These checks MUST be applied regardless of the engine's host OS when the target storage backend is known to be Windows-based (e.g., SMB share on a Windows server).
    SA-STORAGE-18 (ERROR) flags `path:` expressions that incorporate user-controlled input without validation.
28. **Storage TLS enforcement:**
    - **S3:** MUST use HTTPS for all S3 API calls. No HTTP fallback.
    - **FTP:** MUST default to FTPS (implicit TLS on port 990 or explicit TLS via AUTH TLS). Plain FTP is an engine-level configuration option, not a flow-level choice.
    - **SFTP:** Runs over SSH. Host key verification is mandatory (see item 38).
    - **SMB:** SHOULD require SMB 3.x with encryption when traversing untrusted networks.
    - **Local:** No TLS (local filesystem). Engine-level opt-in required (disabled by default).
29. **Storage transfer size limit** -- the engine MUST enforce a configurable maximum transfer size for `storage` operations (default: 1 GB for `transfer`, same as HTTP response limit for `get`). Exceeding raises `ResourceExhaustedError`. Streaming operations (`onYield:`) are exempt from total size but each chunk is subject to per-instance memory limit.
30. **Storage caching conformance levels** -- engines implementing `cacheHint:` support MUST declare one of three conformance levels: **None** (parse the directive, set `RESULT.cache` to `null`, log INFO once per engine lifecycle that caching is not supported), **Basic** (LOCAL scope, TTL-based expiration, EXACT invalidation, `RESULT.cache` populated), **Full** (all `cacheHint:` features including CONTEXT/GLOBAL scope, staleWhileRevalidate, staleIfError, negative caching, warm, PREFIX invalidation, readYourWrites, varyBy, priority). Engines MAY upgrade their conformance level without breaking existing flows. Engines MUST NOT silently downgrade (e.g., claim Full but ignore scope) — unsupported features at the declared level MUST raise `ValidationError` at load time.
31. **Storage cache key construction** -- engines implementing Basic or Full caching MUST construct cache keys using the algorithm: `{scope_namespace}/{resolved_url}/{operation}/{normalized_path}[/{varyBy_result}]`. `scope_namespace` is the flow instance ID for LOCAL, context/correlation ID for CONTEXT, or tenant ID for GLOBAL. `resolved_url` is the fully resolved storage URL (after alias resolution). `operation` is the storage operation name. `normalized_path` is the path after normalization (URL decoding, case normalization per provider). `varyBy_result` is the string result of evaluating the `varyBy:` CEL expression (omitted when `varyBy:` is absent). Cache keys MUST be deterministic — identical inputs MUST produce identical keys.
32. **Storage conditional GET per provider** -- engines implementing CONDITIONAL revalidation MUST use provider-appropriate mechanisms: **S3:** `ETag` + `If-None-Match` / `Last-Modified` + `If-Modified-Since` headers on `GetObject`. **SFTP/FTP:** `stat` to compare `mtime` before full retrieval. **SMB:** `LastWriteTime` attribute comparison. **Local filesystem:** `mtime` + `size` comparison (both must match for cache hit). Engines MUST store the revalidation metadata (ETag, mtime, size) alongside cached content. When the backend confirms no change (S3 304, matching mtime), the engine serves cached content and sets `RESULT.cache.revalidated: true`.
33. **Storage cache scope isolation** -- `scope: GLOBAL` caches MUST be tenant-isolated per ENGINE §5.4 item 8. `scope: CONTEXT` caches MUST be scoped to the context/correlation ID boundary. The engine MUST NOT allow cache entries from one scope to be readable by another scope. Cache key namespace partitioning (item 31) provides the isolation mechanism.
34. **Storage cache warm semantics** -- when `warm: true`, the engine MUST initiate a cache-populating fetch for the specified path at flow load time (before the first step executes). Warm fetches are best-effort: failures are logged at WARN level but do not prevent flow execution. Warm fetches respect `ttl:`, `scope:`, and `maxSize:` from the same `cacheHint:`. Warm fetches for `get` operations retrieve full content; for `info` operations retrieve metadata only. SA-STORAGE-24 (INFO) warns when `warm: true` is used with streaming `get` (`onYield:` present) since the warm fetch loads full content into cache memory.
35. **Storage cache audit logging** -- engines MUST log cache operations at DEBUG level: cache hits, misses, revalidations, invalidations, stale serves, and warm fetches. At INFO level, engines MUST log: cache configuration at flow load time (conformance level, scope), and stale-serve-on-error events (indicating backend failures masked by cache). Cache audit events MUST include: flow instance ID, storage URL (with credentials redacted), operation, path, cache key (truncated to 128 chars), and outcome. Secret redaction rules from §5.4 item 44 apply to all cache log entries.
36. **`SSH` capability** -- deny-by-default. A flow using `ssh` steps MUST declare `requires: { SSH: ... }` with per-alias or per-host command allowlists. The engine MUST resolve `ssh.host:` (bare alias → hostname via engine/parent mapping; literal hostname used directly) and validate the command against the per-host allowlist before execution. There is no host-only form — a command allowlist is always required.
37. **SSH POSIX shell escaping** -- the engine MUST construct the remote command string using POSIX single-quote escaping for all arguments and environment variable values. Each argument is individually escaped as `'<value>'` with internal single-quotes replaced by `'\''`. The engine MUST NOT provide a mechanism to bypass this escaping.
38. **SSH host key verification** -- the engine MUST verify the remote host key for all SSH connections (SSH action and SFTP storage backend). Host key verification cannot be disabled per-flow. Known-hosts management is engine-level configuration. This follows from the TLS verification principle (item 19).
39. **SSH channel restrictions** -- the engine MUST use only the SSH `exec` channel type for `ssh` action steps. `shell` channels (interactive sessions) and port forwarding (`direct-tcpip`, `tcpip-forward`) MUST NOT be supported. No PTY allocation.
40. **Storage and SSH backend registration** -- the engine MUST support two storage backend registration modes: (1) **alias-based** — engine registers named aliases (e.g., `data` → backend config), resolvable by bare name in `url:` field; (2) **URL-prefix-based** — engine maps `scheme://authority` prefixes to credential/backend configurations. Both are engine-level config, opaque to flows. The same dual-mode applies to SSH host registration: alias-based (e.g., `prod_server` → host+credentials) and hostname-based (direct hostname in `host:` field with engine-configured credentials by host pattern). For `file://` URLs, the engine MUST restrict access to configured storage roots; `file://` without explicit engine configuration MUST be rejected.
41. **YAML parsing security** -- The engine MUST use YAML 1.2 safe loading mode (no custom tags, no language-specific type constructors). The engine MUST limit alias expansion: maximum alias count per document (default: 100, max: 1,000), maximum alias nesting depth (default: 10, max: 20). Additionally, the engine MUST enforce a total-expansion-size limit: the fully expanded YAML document MUST NOT exceed 10 MB (configurable, max 100 MB). If expansion exceeds this limit before parsing completes, the engine MUST abort with `ResourceExhaustedError`. The engine MUST enforce a maximum YAML document size (default: 10 MB, same as HTTP response limit). These limits apply to all YAML parsing: flow definition loading, `decode(YAML)`, `parseAs: YAML`, and RESOURCES `.yaml` files. Engine implementations MUST verify YAML 1.2 compliance by running the CROSSLANG.md CL-6 conformance test vectors during CI and in development/test startup modes.
42. **Engine-level interpreter denylist (MUST)** -- The engine MUST provide a configurable global denylist of executable names that are never allowed in `EXEC` or `SSH` command allowlists, regardless of flow declarations. Default denylist (MUST ship with these entries, operators MAY extend): `bash`, `sh`, `zsh`, `fish`, `dash`, `ksh`, `csh`, `tcsh`, `cmd`, `cmd.exe`, `powershell`, `pwsh`, `powershell.exe`. Interpreters like `python3`, `node`, `ruby`, `perl`, `php` MUST be flagged with a mandatory acknowledgment in engine configuration when added to any allowlist — the engine MUST NOT silently allow these without explicit operator opt-in. SA-EXEC-10 (ERROR) and SA-SSH-3 (ERROR) enforce at the flow level; this item enforces at the engine level as defense-in-depth. Denylist matching MUST be case-insensitive and MUST match with or without platform-specific file extensions (e.g., `.exe`, `.cmd` on Windows). `cmd`, `cmd.exe`, and `CMD.EXE` all match the same denylist entry.
43. **EVENT.SOURCE assignment** -- The engine MUST assign `EVENT.SOURCE` using the canonical flow identity (flow path + instance UUID). This field is immutable — no flow code may modify it. Source filtering MUST be evaluated at the engine level before event delivery, not at the application level.
    **Event data size limit.** The engine MUST enforce a configurable maximum event data payload size (default: 1 MB per event, configurable maximum: 10 MB). Events exceeding the limit MUST be rejected with `ResourceExhaustedError` at emission time. *(CWE-400)*
44. **URL secret redaction** -- When the engine constructs an HTTP request URL and any query parameter value was resolved from a `SecretValue` handle, the engine MUST replace that parameter's value with `[REDACTED]` in all logged/traced representations of the URL. The engine tracks SecretValue provenance through URL construction — it does not scan URL strings for secret patterns (which would be fragile and risk false negatives).
45. **Schema fetch integrity** -- The engine SHOULD provide `schemaFetch.requireIntegrity` configuration (default: `false`, recommended `true` for production). When enabled, `$ref` URLs without `integrity:` are rejected at load time.
46. **CSV parsing limits** -- The engine MUST enforce the following limits for all CSV/TSV parsing (`parseAs: CSV`, `parseAs: TSV`, `decode(CSV)`, `decode(TSV)`, and RESOURCES `.csv`/`.tsv` loading): maximum row count (default: 100,000), maximum column count (default: 1,000), maximum cell size (default: 1 MB). Exceeding raises `ResourceExhaustedError`. The engine MUST sanitize CSV/TSV output that contains formula-prefix characters (`=`, `+`, `@`, `-`) in cell values by prepending a single quote (`'`) or tab character before the formula-prefix character when the cell value starts with one of these characters. This prevents CSV injection (CWE-1236) when downstream consumers are spreadsheet applications. The sanitization applies to `encode(CSV)`, `encode(TSV)`, and any engine API that produces CSV/TSV output. Additionally, action providers that produce CSV or TSV output intended for downstream spreadsheet consumption MUST apply the same formula-prefix sanitization. This requirement extends to any action provider whose output contract declares `format: CSV` or `format: TSV`. SA-CSV-1 (WARN) flags formula-prefix characters detected at static analysis time in headers and literal values.
47. **HTTP auth TLS enforcement** -- The engine MUST provide `request.requireTlsForAuth` configuration (default: `true`). When enabled, `auth:` (basic or bearer) over `http://` is rejected at runtime, except when the resolved IP address is a local address (`127.0.0.0/8`, `::1`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`). The local address exemption allows development and internal-service use cases without TLS. The engine MUST provide a `request.requireTlsForAuthIncludingLocal` override (default: `false`) that, when enabled, enforces TLS even for local addresses.
48. **SRI hash format** -- SRI hash format: `{algorithm}-{base64-hash}` where algorithm is `sha256`, `sha384`, or `sha512`. Only one hash per `integrity:` field (FlowMarkup uses single-hash for simplicity since the spec controls both producer and consumer). Hash comparison MUST be constant-time to prevent timing attacks. **Future consideration (v1.0+):** Support multi-hash SRI for hash agility: `integrity: sha256-<base64> sha384-<base64>`. When multiple hashes are provided, verification succeeds if ANY hash matches (allows migration when an algorithm is deprecated). Engine implementations SHOULD design their integrity verification to accommodate future multi-hash support without breaking changes.
49. **Cumulative memory tracking** -- The per-instance memory limit (configurable) applies to the sum of all concurrent response bodies, decompressed content, CEL evaluation buffers, and transaction fork memory. The engine MUST track cumulative memory, not just per-response.
50. **Remote flow integrity (MUST for production)** -- The engine MUST provide a configuration option to require `integrity:` on all remote flow references (strict mode). This option MUST default to `true` for all deployment profiles. When strict mode is disabled (operator override), the engine MUST log a WARNING at startup: 'Remote flow integrity verification disabled — remote flows will execute without hash verification.' This log entry MUST be emitted unconditionally (cannot be suppressed by log level configuration). When strict mode is enabled, remote flow references without `integrity:` MUST be rejected at load time with `ValidationError`. SA-RUN-2 (ERROR) enforces at the flow level; this item enforces at the engine level.
51. **ERROR.DATA secret redaction** -- The engine MUST apply secret redaction to `ERROR.DATA` fields before making error data available to flow `catch:` handlers. Fields derived from resolved `SecretValue` handles MUST be replaced with `[REDACTED]`. This prevents secret leakage through action provider error responses that echo back request headers or bodies containing resolved secrets (CWE-209). SA-SECRET-27 (ERROR) flags error schemas with reflected input fields at static analysis time; this item enforces at runtime.
52. **HTTP header CRLF validation** -- The engine MUST validate resolved `SecretValue` content before injection into HTTP headers — values containing CR (`0x0D`) or LF (`0x0A`) MUST be rejected with `ValidationError`. This prevents HTTP header injection (CWE-113) via malicious secret values. Modern HTTP clients typically reject CRLF in headers by default; this requirement provides defense-in-depth at the engine level.
53. **ENV credential output redaction (MUST)** — In addition to Item 7 (log/trace redaction), the engine MUST redact `ENV.*` values whose keys match credential patterns (`PASSWORD`, `SECRET`, `TOKEN`, `API_KEY`, `PRIVATE_KEY`, `CREDENTIAL`, `AUTH` — case-insensitive) from all error messages and diagnostic output visible to flow authors or end users. This complements Item 7's log-level redaction with application-level output protection.
54. **Secret rotation grace period (MUST)** — Engines implementing automatic secret rotation MUST support a configurable grace period (minimum 30 seconds, default 60 seconds) during which both previous and current secret values are valid. During the grace period: the engine MUST resolve the current value for new requests; action providers MUST accept both current and previous values; the engine MUST emit an audit event when a previous-version secret is used successfully; after the grace period expires, the engine MUST revoke the previous value.
55. **`has(SECRET.*)` constant-time evaluation (MUST)** — The engine MUST evaluate `has(SECRET.name)` in constant time regardless of whether the secret exists, is accessible, or the name is valid. Implementation: pre-load the set of accessible secret names at flow initialization; `has()` checks membership in this pre-loaded set using a constant-time lookup (e.g., hash set). The response time MUST NOT vary by more than 10% between existing and non-existing secret names. Additionally, engines MUST add CSPRNG jitter per [FLOWMARKUP-SECRETS.md](FLOWMARKUP-SECRETS.md) §9.
56. **Secret memory lifecycle (MUST)** — After action execution completes and the resolved secret value is no longer needed, the engine MUST clear the secret value from memory using platform-appropriate mechanisms (see [FLOWMARKUP-ENGINE-CROSSLANG.md](FLOWMARKUP-ENGINE-CROSSLANG.md) CL-14). In GC-managed languages, engines MUST minimize secret value lifetime and avoid creating unnecessary copies. Engines MUST NOT cache resolved secret values beyond the action execution boundary unless the secret provider explicitly supports caching with a declared TTL.
57. **Event source authentication (MUST)** — External event sources (webhooks, message queues, HTTP triggers) MUST be authenticated by the engine before delivery to flow triggers. The authentication mechanism is engine-configured and MUST support at least one of: HMAC signature verification (e.g., `X-Hub-Signature-256`), OAuth2 bearer token validation, or mTLS client certificate verification. Unauthenticated events MUST be rejected with HTTP 401/403 and MUST NOT be delivered to any flow trigger. The engine MUST log rejected events as audit events including source IP and event type.
58. **Async security error counter (MUST)** — The engine MUST maintain a per-flow-instance counter of security errors from async sub-flows (`MissingCapabilityError`, `SecretAccessError`, `AuthenticationError`, `AccessDeniedError`). This counter is exposed via `RUNTIME.ASYNC_SECURITY_ERRORS` (requires `RUNTIME` capability). The counter is append-only and monotonically increasing. This provides flow authors programmatic access to detect async security violations, complementing the unconditional audit logging in Item 20.

59. **Process isolation.** The engine MUST isolate concurrent flow instances such that one instance cannot read or modify the memory, variables, or intermediate state of another instance. Implementations MUST use at minimum thread-level isolation with no shared mutable state between flow instances; process-level or container-level isolation is RECOMMENDED for multi-tenant deployments. *(CWE-668: Exposure of Resource to Wrong Sphere)*

60. **Parallel branch limits.** The engine MUST enforce a configurable maximum number of concurrent branches within a single `parallel` block (default: 64, configurable maximum: 1024). Exceeding the limit MUST raise a `ResourceLimitError`. The engine MUST also enforce a per-tenant aggregate parallel branch limit (default: 512) to prevent resource exhaustion across multiple flow instances. *(CWE-770: Allocation of Resources Without Limits or Throttling)*

61. **Fail-closed engine boundary.** When the engine encounters an internal error (out of memory, uncaught exception, storage backend failure, checkpoint corruption), it MUST halt the affected flow instance and transition it to a FAILED state rather than continuing execution with potentially corrupted state. Engines MUST NOT silently swallow internal errors or allow execution to proceed when security-critical invariants (capability enforcement, secret redaction, taint tracking) cannot be guaranteed. *(CWE-636: Not Failing Securely)*

62. **ERROR.CAUSE chain depth limit.** The engine MUST enforce a maximum ERROR.CAUSE chain depth (default: 16, configurable maximum: 64). When the limit is reached, the engine MUST truncate the oldest cause entries and replace them with a single summary entry indicating truncation occurred. Before exposing ERROR.CAUSE to catch handlers, the engine MUST apply the same secret redaction rules as ERROR.DATA (item 51). *(CWE-209: Generation of Error Message Containing Sensitive Information)*

63. **Distributed execution security.** Engines that support distributed execution across multiple nodes MUST implement the following security requirements:
    - **Mutual TLS between nodes:** All inter-node communication (control plane, data plane, and health checks) MUST use mutual TLS (mTLS) with certificate-based identity. Each engine node MUST present a valid client certificate; connections from nodes without valid certificates MUST be rejected.
    - **Encrypted in-transit flow state:** Flow state (checkpoints, variable snapshots, continuation data) transferred between engine nodes MUST be encrypted in transit. The engine MUST use TLS 1.2+ for all inter-node state transfer. Unencrypted inter-node flow state transfer MUST NOT be supported, even in development/test configurations.
    - **Node authentication for work units:** Before accepting a work unit (flow instance assignment, branch delegation, checkpoint transfer), a node MUST authenticate the sending node's identity against a trusted node registry. Work units from unauthenticated or deregistered nodes MUST be rejected and logged as audit events.
    - **Consensus:** State transitions for GLOBAL variables and cross-instance locks MUST use a consensus protocol that tolerates at least one node failure (e.g., Raft, Paxos).
    - **Orphaned lock detection:** The engine MUST detect locks held by terminated or crashed flow instances and release them within a configurable detection interval (default: 30 seconds, maximum: lock TTL).
    - **Split-brain protection:** The engine MUST detect network partitions and halt affected flow instances rather than allowing divergent execution. Partitioned instances MUST transition to a SUSPENDED state with automatic reconciliation on partition heal.
    - **Event delivery:** Cross-node event delivery MUST provide at-least-once semantics with idempotent processing. The engine MUST authenticate inter-node communication using mutual TLS (mTLS) or equivalent channel-level authentication.
    - **Clock synchronization:** Distributed lock TTLs and timeout calculations MUST account for clock skew. Engines MUST use monotonic clocks for local timing and MUST use logical clocks (vector clocks or hybrid logical clocks) for distributed ordering. Wall-clock skew can cause lock expiration races leading to concurrent access to protected resources. *(CWE-362: Concurrent Execution Using Shared Resource with Improper Synchronization)*

64. **Secret provider SSRF protection.** Secret provider communication (Vault, AWS Secrets Manager, etc.) MUST be subject to the same SSRF protections as `request:` actions: scheme allowlisting (HTTPS only), DNS pinning, private IP validation. Secret provider URLs MUST be validated at engine startup and MUST use HTTPS. *(CWE-918)*

65. **`forEach` concurrent execution bound.** When `forEach` uses `concurrent: true` (with `maxConcurrency`), the engine MUST limit concurrent iterations to the same configurable maximum as `parallel` block branches (item 60, default: 64). Iterations beyond the concurrency limit MUST be queued, not rejected. The per-tenant aggregate parallel branch limit (item 60) applies across both `parallel` blocks and concurrent `forEach` iterations. *(CWE-770)*
66. **Single-step `GLOBAL.*` read-modify-write atomicity (MUST).** When a single `set:` step reads one or more `GLOBAL.*` keys and writes back to any `GLOBAL.*` key, the engine MUST guarantee atomicity of the entire read-modify-write sequence using optimistic concurrency control (OCC) or compare-and-swap (CAS). The engine MUST: (1) capture the version/ETag of each `GLOBAL.*` key read during CEL expression evaluation; (2) at write time, verify that no captured key has been modified since the snapshot read; (3) if a conflict is detected, transparently retry the entire step (re-read + re-evaluate + re-write) up to a configurable maximum (default: 100 retries, configurable maximum: 1,000); (4) if retries are exhausted, raise `ConflictError`. This guarantee applies per-step outside of `transaction: true` groups — inside transaction groups, the transaction-level serializable isolation (FLOWMARKUP-SPECIFICATION.md §2.10 "Transaction Group Isolation") subsumes this requirement. The OCC/CAS mechanism MUST be tenant-isolated (item 8) — version checks MUST NOT cross tenant boundaries. *(CWE-362: Concurrent Execution Using Shared Resource with Improper Synchronization)*

### 5.5 Flow Identity and Versioned Execution

Every flow has a **FlowRef**: `(canonicalLocation, contentHash)`.

**Content hash:** SHA-256 of raw file bytes, encoded as `sha256-<base64>`. Computed once at load time from original source bytes.

**Flow cache:** MUST be keyed by `(canonicalLocation, contentHash)`. Cache lookup: fetch -> compute hash -> verify integrity -> lookup -> parse if miss.

**Remote flow fetch cache:** For remote flows (HTTP/S, git-backed), the engine MAY cache raw fetched bytes to avoid repeated network requests when the same remote flow is referenced by multiple sub-flows within or across executions.

- **Policy:** Engine configuration sets explicit TTLs. When no engine TTL is configured, HTTP response hints (`Cache-Control`, `Expires`) are taken as advisory inputs. Engine configuration always takes precedence over HTTP hints.
- **Revalidation:** Engine SHOULD support conditional HTTP requests (`ETag` / `If-None-Match`, `Last-Modified` / `If-Modified-Since`) to revalidate stale entries without re-downloading unchanged content.
- **No version pinning for sub-flow resolution:** The engine MUST NOT pin a specific version of a remote sub-flow for the lifetime of a parent flow. Each sub-flow invocation resolves the flow independently based on the fetch cache state at call time. If the cache has expired between two calls to the same sub-flow URL, the second call may resolve to a newer version than the first — this is intentional and expected. Long-running flows (e.g., waiting days for events) MUST be able to pick up updated sub-flow versions as cache entries expire, rather than being forced to use a stale version pinned at the start of execution.
- **Instance version pinning:** While sub-flow *resolution* is not version-pinned (see above), a flow instance's own code IS pinned. Once a flow instance begins execution, the engine MUST use the same `contentHash` version of that flow's code throughout the instance's entire lifecycle — running, paused, waiting for events, and resumed. The engine MUST durably persist the flow code itself (not just the content hash) for all active instances, so that resumption does not depend on fetch cache state or remote source availability. The controlled exception is crash recovery: when the engine detects a newer version is available and the flow defines `onVersionChange:`, it may migrate the instance to the new version (see §5.5 Persistence and crash recovery).
- **Root-flow freshness:** When a flow is invoked directly as a root (not as a sub-flow), the engine SHOULD prefer a fresh version — bypass or revalidate the fetch cache for the root flow and its local sub-flows — unless configured otherwise. Remote sub-flows follow normal cache policy.
- **Cache management:** The engine MUST expose operations to invalidate the fetch cache for a specific flow URL and to clear the entire fetch cache. Invalidation takes effect on the next fetch of the affected flow.

**Persistence and crash recovery:** Persisted state MUST include `FlowRef` of every flow in the call stack. The engine MUST durably persist the flow code alongside checkpoint state for all active instances. Persisted checkpoint data MUST be encrypted at rest using authenticated encryption (e.g., AES-256-GCM). The encryption key MUST be managed at the engine level (not derived from flow data). Key management requirements for checkpoint encryption:
- The encryption key MUST be derived using a KDF (HKDF-SHA256 recommended) from a master key, with the flow instance ID as context/info parameter. The engine MUST generate a random per-checkpoint salt (minimum 16 bytes) and include it in the HKDF context alongside the instance ID. The Data Encryption Key (DEK) MUST be randomly generated, not derived solely from the instance ID. The salt MUST be stored alongside the encrypted checkpoint.
- Key rotation MUST be supported without re-encrypting existing checkpoints (use envelope encryption: each checkpoint encrypted with a DEK, the DEK encrypted with the current KEK)
- The master key SHOULD be stored in an HSM, KMS, or equivalent hardware-backed key store. Engines that do not support HSM MUST document this limitation and log a WARNING at startup
- The engine MUST NOT store encryption keys in the same storage system as the encrypted checkpoints
The engine MUST enforce tenant isolation in checkpoint storage — a flow instance MUST NOT be able to access checkpoint data belonging to a different tenant. The engine MUST verify checkpoint integrity on resume using the authenticated encryption tag (or a separate HMAC if using a non-AEAD cipher) to detect corruption or tampering. Checkpoint data contains flow variable state which may include credentials, PII, or business-sensitive data — encryption at rest is mandatory, not advisory.

On resume, the engine checks for updates (re-fetches or checks fetch cache) and resolves as follows:

- Same content hash → resume with persisted code directly
- Content changed with `onVersionChange:` → execute migration handler. Three outcomes: (1) handler completes normally → resume with new version; (2) handler executes `return` → gracefully terminate instance (`finally:` runs, checkpoint discarded, return value becomes flow output); (3) handler throws → `MigrationError` (checkpoint preserved for retry). This is the designed upgrade path for long-running instances — flow authors opt in to safe version migration by providing the handler.
- Content changed without `onVersionChange:` → `FlowVersionError`. The engine MUST NOT silently use either the old or new version when no migration path is defined, as checkpoint state may be incompatible with the new version.

During `onVersionChange:` handler execution, the effective capabilities MUST be the intersection of the old version's `requires:` and the new version's `requires:`. This ensures the migration handler cannot exercise capabilities that either version lacks. If the new version removes a capability that the handler attempts to use, the engine raises `MissingCapabilityError` — the handler SHOULD catch this and degrade gracefully (e.g., skip capability-dependent cleanup) or terminate the instance via `return`.
- Remote unreachable, persisted code available → resume with persisted code (the content hash has not changed from the engine's perspective)
- Remote unreachable, persisted code unavailable → `FlowVersionError`

`FlowVersionError` and `MigrationError` are non-catchable engine-level errors.

> **Cryptographic agility.** Implementations MUST support algorithm negotiation for checkpoint encryption to enable migration to post-quantum cryptographic algorithms. The checkpoint envelope format MUST include an algorithm identifier field. When NIST post-quantum standards (FIPS 203 ML-KEM, FIPS 204 ML-DSA) reach broad library availability, engines SHOULD offer hybrid classical+PQ encryption modes (e.g., AES-256-GCM with ML-KEM-768 key encapsulation) as a configurable option. Implementations MUST use AES-256 (not AES-128) for symmetric encryption of checkpoints to maintain adequate security margin against quantum key search (Grover's algorithm). *(See NIST IR 8547 transition timeline.)*

### 5.6 Observability

Each step creates an OpenTelemetry span:
- Span name: `{type}:{_label_}` or `{type}[{index}]`
- Groups: attribute `group.mode`. Branches create child spans.
- Error spans include `ERROR.TYPE`, `ERROR.MESSAGE`
- Sub-flow `run` spans SHOULD emit `flow.integrity`
- Root flow spans MUST include `flow.location` and `flow.integrity`

### 5.7 Audit Logging

For compliance-sensitive deployments (payment processing, claims workflows, regulatory processes), the engine MUST support a tamper-evident audit log. The audit log is separate from OpenTelemetry traces and provides a durable, append-only record of security-relevant events.

**Mandatory audit events (MUST log when audit logging is enabled):**

| Event | Data |
|---|---|
| Flow invocation | FlowRef, caller identity, input hash, timestamp |
| Secret access | Secret name, flow identity, access type (resolve/has-check), timestamp |
| Capability grant | Parent flow, child flow, requested capabilities, available capabilities, granted capabilities (intersection result), denied capabilities (if any), timestamp |
| GLOBAL.* write | Key, old value hash, new value hash, flow identity, timestamp |
| exec invocation | Command, args (redacted), flow identity, timestamp |
| mail send | Recipients, subject, flow identity, timestamp |
| request outbound | Method, URL (redacted query), flow identity, timestamp |
| storage operation | Resolved URL (or alias), operation, path(s), flow identity, timestamp |
| ssh invocation | Resolved host (or alias), command, args (redacted secrets), flow identity, timestamp |
| Error (unhandled) | Error type, flow identity, step _id_, timestamp |
| Async security error | Error type, child FlowRef, parent FlowRef, parent step _id_, capability involved, timestamp |
| onVersionChange execution | Old FlowRef, new FlowRef, variable diff (name, old/new value hash), deployer identity, timestamp |

**Capability evaluation audit:** When a `run:` step evaluates capabilities, the engine MUST include in the audit event: `requested` (sub-flow's `requires:`), `available` (caller's effective capabilities), `granted` (intersection result), `denied` (requested minus granted, if any). When `MissingCapabilityError` is raised, the audit event MUST include the specific missing category and entries.

**Audit log format.** Audit log entries MUST be serialized as JSON objects, one entry per line (JSON Lines / NDJL format). Each entry MUST contain at minimum: `timestamp` (ISO 8601 with millisecond precision in UTC, e.g., `"2025-03-15T14:30:22.456Z"`), `event_type` (string from the event table above, e.g., `"flow.invocation"`, `"secret.access"`, `"global.write"`, `"exec.invocation"`, `"mail.send"`, `"request.outbound"`, `"storage.operation"`, `"ssh.invocation"`, `"error.unhandled"`, `"async.security_error"`, `"version.change"`), `flow_id` (fully-qualified flow identifier), `tenant_id`, `instance_id`, `engine_id`, `severity` (one of `INFO`, `WARN`, `ERROR`, `CRITICAL`), and `chain_hash` (SHA-256 hash for tamper-evident chaining, hex-encoded). Event-specific fields are nested under a `data` key. Engines MUST validate that all timestamps use UTC (suffix `Z`) — local time offsets are NOT permitted in audit entries. Engines MUST include a `schema_version` field (currently `"1.0"`) to enable forward-compatible format evolution.

**Properties:**
- **Append-only:** Audit entries MUST NOT be modifiable or deletable through the engine API.
- **Tamper-evident:** Engines that enable `EXEC`, `SSH`, `MAIL`, or `STORAGE` capabilities MUST implement cryptographic log chaining: each audit log entry includes a hash of (`previous_entry_hash` + `current_entry_content`) using SHA-256. The chain anchor (first entry) uses a zero hash. This enables tamper detection for security-critical audit trails. Engines without these capabilities SHOULD implement chaining.
- **Redaction:** Secret values MUST be redacted from audit entries. Only secret names and access patterns are logged.
- **Retention:** Engine MUST support configurable retention policies.

Audit logging is **MUST** for engines that enable any of: `EXEC`, `MAIL`, `SECRET`, `GLOBAL` write, `STORAGE`, or `SSH` capabilities. Engines that only support read-only flows without these capabilities MAY omit audit logging. The engine MUST NOT provide a mechanism to disable audit logging when EXEC, MAIL, SECRET, GLOBAL write, STORAGE, or SSH capabilities are provisioned.

**Audit log availability (fail-closed).** When the audit log storage becomes unavailable or full, the engine MUST halt affected flow instances that require audit logging (flows with `SECRET`, `EXEC`, `SSH`, or `STORAGE` capabilities) rather than continuing execution without audit coverage. Flows without security-sensitive capabilities MAY continue with a logged WARNING. The engine MUST expose an audit-log-health metric and SHOULD support configurable alerting when audit logging degrades. *(CWE-778: Insufficient Logging)*

**Unconditional async security events:** Security errors (`MissingCapabilityError`, `SecretAccessError`, `AuthenticationError`, `AccessDeniedError`) from `async: true` sub-flows MUST be logged even when audit logging is globally disabled. These events represent security boundary violations that would otherwise be invisible — the caller receives no error (fire-and-forget), and without unconditional logging the violation may go entirely undetected. See §5.4 item 20.

> **Post-quantum hash chain readiness.** The cryptographic log chaining mechanism MUST support configurable hash algorithms. While SHA-256 provides adequate preimage resistance post-quantum, for new deployments, SHA-384 is RECOMMENDED as the default hash algorithm; existing deployments SHOULD plan migration to SHA-384 or SHA-3-256 for long-lived audit trails where collision resistance is critical. The log chain format MUST include an algorithm identifier to enable future hash algorithm rotation without breaking chain verification.

### 5.8 Cross-Language Implementation Guidance

The normative requirements in this document must be translated to each engine's host language. The companion document [FLOWMARKUP-ENGINE-CROSSLANG.md](FLOWMARKUP-ENGINE-CROSSLANG.md) provides:

- **Cross-language guidance items:** Concise translation tables for numeric types, sum types, concurrency primitives, plugin discovery, CEL libraries, YAML parsers, introspection denylists, XML security, error hierarchies, charset detection, JSON Schema validation, content hashing, regex engine selection, secure memory clearing, Java deserialization safety, Python pickle prohibition, prototype pollution mitigation, C# SecureString deprecation, and Rust unsafe FFI guidance across Java, Go, Python, TypeScript, Rust, and C#.
- **Known ecosystem gaps** (e.g., Rust YAML 1.2 parser fragmentation, C# CEL immaturity) with suggested workarounds.
- **Conformance test vectors** for cross-language interoperability validation.

All library suggestions in that document are non-normative; implementors MUST validate dependencies independently.

---

## 6. Tooling -- Flow File URL Resolution

Tools that load flow files from URLs MUST normalize Git hosting UI URLs to raw-content URLs:

| Input Pattern | Normalized To |
|---|---|
| `https://github.com/{user}/{repo}/blob/{ref}/{path}` | `https://raw.githubusercontent.com/{user}/{repo}/{ref}/{path}` |
| `https://gitlab.com/{ns}/-/blob/{ref}/{path}` | `https://gitlab.com/{ns}/-/raw/{ref}/{path}` |
| `https://bitbucket.org/{user}/{repo}/src/{ref}/{path}` | `https://bitbucket.org/{user}/{repo}/raw/{ref}/{path}` |
| Any other URL | Used as-is |

Query strings and hash fragments are always stripped.

---

## 7. Testing

FlowMarkup provides a native YAML-first testing framework. Tests are authored as `.flowmarkup-test.yaml` files.

See **[FLOWMARKUP-TESTING.md](FLOWMARKUP-TESTING.md)** for the full specification covering test file format, test tiers (UNIT/INTEGRATION), mocks, fixtures, assertions, parameterized tests, migration testing, snapshot testing, invariants, contract testing, lifecycle hooks, fault injection, step-level testing, coverage, concurrency/timing, and SA-TEST rules.