Architecture
End-to-end Cascades platform architecture—UI through API, queue, workers, execution engine, proofs, transparency logs, anchoring, and storage.
Cascades separates authoring (what should run), execution (what actually ran), reliability (retries, queues, DLQ), and trust (snapshots, proofs, transparency, optional blockchain anchoring). This page is the fastest way to understand how those layers connect without rereading every feature guide.
Orchestration flow
- Author a workflow definition in the catalog or UI (DAG of tasks, edges, policies).
- Validate & snapshot — accepted definitions persist as immutable snapshots used for enqueue and audit (immutable snapshots).
- Enqueue — API/UI schedules a workflow run: work lands on Redis-backed queues.
- Workers pull jobs, hydrate context, and hand executable units to the execution engine which resolves DAG order, branching, retries, and timeouts (DAG execution, retries & timeouts).
- Outputs & state persist to DB (run/task state, metadata) and object storage (artifacts where applicable).
This is the path from intent (definition) → scheduled work → deterministic orchestration.
Reliability flow
Operational robustness hangs on queues, backoff, and observability—not on “hope the API stays up.”
- Queues isolate spikes: API acknowledges scheduling; workers drain at a sustainable rate (queue workers).
- Retries apply per-task policies; timeouts prevent poison work from wedging downstream (retries & timeouts).
- Dead-letter / replay paths capture terminal failures for investigation (failure recovery).
- Concurrency & locking behavior (deployment-specific) avoids double execution of exclusive sections.
Trust flow
Cascades is designed so audits can answer: “What ran, under what definition, with what evidence?”
- Snapshot hashes the enqueue-time structural definition independently of mutable UI state.
- Execution proofs capture canonical payloads linking run identity, task outcomes, and policy metadata (execution proofs).
- Optionally register proof digests with Rekor-aligned transparency tooling for third-party corroboration (Rekor).
- Optional blockchain anchoring roots transparency or checkpoint commitments on-chain depending on deployment policy (blockchain anchoring).
- Drift detection contrasts enqueue-time fingerprints with evolving live definitions (drift detection).
Near-term replay will close the loop between proofs and deterministic re-execution (replay roadmap).
Storage flow
- Transactional state — run progress, schedules, proofs metadata, tenancy records.
- Blobs / artifacts — logs, payloads, exporter outputs referenced from runs where product configuration allows.
- Cache / coordination — Redis (or deployment equivalent) backs fast queue semantics and cooperative locks—not the system of record for durable audit history.
Where to go next
- Core concepts — vocabulary: DAGs, tasks, dependencies.
- Workflow lifecycle — gates from draft to audited runs.
- Deployment models — topology for operators.
- Verification — proof-first posture in depth.