Architecture

How the factory is built, at the level you need before you change it: what it is for, what it connects to, what runs where, and the two rules everything else answers to. Each page after this one goes a level deeper.

At the centre of all of it is the data contract: v0.1 when Source ends, 0.x while it is agreed in Refine, 1.0.0 once it goes live — one artefact, called the same in the interface and in the code.

Goals and constraints

Five quality goals, in this order. Quality and risks gives each one's cost, and the tests that show it holds.

1. Auditability. One record; every decision on it named and timed; nothing in it edited or deleted once written.
2. Determinism. The same contract produces the same SQL, with no model in between.
3. Failing closed. Told two contradictory things, or too little, the factory refuses rather than guesses.
4. Portability across warehouses. Another engine is a connection, not a rewrite.
5. Operability on one small box. One person can run it: one service, one database file, no build step.

The constraints, each with the decision that set it:

Constraint	Why
Server-rendered Flask and Jinja; no build step, no client framework	Decision 0003
The record is a database of its own, apart from the warehouse	Decision 0002
The contract is the source of truth; SQL is derived from it	Decision 0001
The process is a validated file, not code	Decision 0004
Agents propose; named people decide	Decision 0005
The API reads and never writes	Decision 0006

In context

People work in the application, each on their own part of the record. Systems and agents read the same record — over the API with a service token, or over MCP — and cannot change it. Everything on the right is a connection, set in configuration: with none of them set, the factory still runs on its own — nobody signs in, DuckDB is the warehouse, and the catalogue is a file.

What runs where

One web application, one command line, two databases.

The web application is Flask under gunicorn, two workers, and it serves everything that answers HTTP: the pages, this documentation, the read API at /api/v1, and MCP over HTTP at /api/v1/mcp. The command line, python -m dpf, is where an operator administers the factory and cuts a release of it, and where an agent on the same machine reaches MCP on stdio.

Both run the same application code. The work that spans systems — drafting with an agent, building and testing in the warehouse, cutting a release, promoting, rolling back, detecting drift — is a function in pipeline.py, called the same way from a page and from the shell, so the record tells one story whichever way the work came in. Most reads and writes of the record are in repo.py, and the rules are in domain/, which touches neither database. Under them are the two databases — the control plane, which is the record, and the warehouse, which is the data — and the outside services the factory calls when they are configured. The identity provider is the one outside service only the web application talks to; see Identity.

Why the record is apart from the data is on The record; how the warehouse stays swappable is on The warehouse.

The two invariants

Everything here is answerable to these two rules. A change that would break one is the wrong change.

1. The contract is the source of truth; the SQL is derived and regenerable. Given a contract, domain/generate.py produces byte-identical SQL every time, with no model involved. That is what makes an audit trail mean something: the question "what produced this table?" has one answer, and it can be checked. Break this and the factory is a chatbot that writes SQL.

2. Every shipped column and every test traces to a versioned requirement. Columns declare satisfies, expectations declare verifies, and both live inside the hashed contract body, so one file is self-contained evidence. A criterion that nothing verifies is reported as uncovered rather than passing quietly. Break this and it is a build tool, not a control plane.

A release keeps the SQL it was built from, and going live runs that stored text rather than regenerating it; a rollback is a new promotion of an earlier release, never the deletion of the one it replaces. Runtime follows both in order.

The process is configuration

dpf/workflow.yaml names the phases, their steps, the gate that ends each phase and who is accountable for it — the drawing on the overview is generated from it. The application validates the file when it starts and refuses a step that does not exist or a dependency in the wrong order, rather than rendering a process that is subtly wrong. DPF_WORKFLOW points at another file to run another process: see the workflow reference, and Governance for why the three gates ask three different questions of three different people.

In this section

Page	What it answers
Components	The building blocks, what each may and may not do, and where a change goes
The record	What the factory stores, what it computes instead, and what never changes
Runtime	The scenarios that matter, step by step, in the order they happen
Deployment	What runs on the server, and what leaves it
Identity	The doors, and how a person becomes a role
Security model	What the controls protect, and how they fail closed
The warehouse	The seam between the factory and the engine that holds the data
Crosscutting concepts	The rules that apply everywhere
Versioning	How the factory and these pages are versioned together
Quality and risks	What the design optimises for, and what it still owes

Why it is this way, decision by decision, is in the decision records.