Deployment

One Linux server runs the whole factory: nginx at the edge, one gunicorn process behind it, and one directory both may write to. Outside that box are the handful of systems it can be configured to reach, and the two numbers that say which version and which exact commit are running.

On the box

nginx is the only thing the network ever reaches. It terminates TLS and proxies everything under the configured prefix (DPF_PREFIX, typically /dataproductfactory) straight to gunicorn on 127.0.0.1, loopback only — binding the public interface instead would quietly run a second, unencrypted copy of the site next to the one nginx is protecting. proxy_pass carries no trailing slash on purpose. With one, nginx would strip the matched location before forwarding it on; without one, the request arrives with the prefix still on it, and _Mounted (dpf/web/__init__.py) does the stripping itself, moving the prefix onto SCRIPT_NAME so every url_for call still builds a link with the prefix already on it. ProxyFix then trusts exactly one hop of X-Forwarded-For and X-Forwarded-Proto, so the application sees the visitor's real address and https, never nginx's own loopback connection.

The location block sets no Cache-Control of its own: the application decides, response by response — private, no-store on every page and on the API, a year on a static file asked for by its own hash (static assets). proxy_no_cache and proxy_cache_bypass keep nginx itself from caching anything, because almost everything behind the prefix depends on who is asking. It also caps a request body at 4 MB, for an uploaded section of a data contract v0.1; the application's own cap (MAX_CONTENT_LENGTH, dpf/web/__init__.py) is stricter, at 2 MB, so anything in between the two reaches the application and is refused there, by its own 413 handler, rather than by nginx.

Behind nginx, one gunicorn process runs two workers under systemd, started as wsgi:app — wsgi.py's only job is calling create_app() once, under the name gunicorn was told to import. Type=notify means systemd waits for gunicorn's own readiness signal before marking the unit active, rather than reporting a restart "up" the instant the process starts and before a worker has actually bound the port. The rest of the unit is the hardening any small service on a shared box should have, each line answering one question: NoNewPrivileges stops the process gaining anything it did not start with; PrivateTmp gives it a /tmp nothing else on the box can see into; ProtectSystem=strict makes the whole filesystem read-only to it; ProtectHome removes every real user's home directory from view; and ReadWritePaths is the one exception carved out of all of that — var/, and nothing else. Restart=on-failure with RestartSec=3 is a different kind of safety: gunicorn comes back a few seconds after dying, rather than leaving the prefix answering nothing until somebody notices.

The environment file (/etc/dataproductfactory.env) is root-owned and mode 600, and the unit loads it with EnvironmentFile= rather than holding the values itself, because a unit under /etc/systemd is world-readable — a secret written into it would not be a secret.

var/ is the one path ReadWritePaths names, and the only thing the application ever writes to disk: the control-plane SQLite file, and, when the warehouse is DuckDB rather than Databricks, the warehouse file beside it.

Mounted inside another site rather than run on its own domain, DPF_PREFIX is what makes url_for build every link with that prefix already on it. SECRET_KEY has to be the host site's own key in that arrangement: reading its session cookie means signing with the same key, and a different one here would not just fail to read it — it would sign the host site's own visitors out. See Identity for what that session cookie goes on to mean.

What leaves the box

Unconfigured, the factory reaches nothing outside itself: no door, DuckDB in var/, a bundled JSON catalogue, and the agents unavailable. Each of the following is a separate connection, made only once its own settings are set — never because another one already is.

What ships, and which version is running

A deployment is the application's own code, its templates and static files, the documentation, and two small files at the repository root: VERSION and BUILD. version.current() (dpf/version.py) reads VERSION once per process — a deploy restarts gunicorn, so reading it twice would only ever answer the same question twice — and the sidebar, the CLI, the API and the documentation's own header all print it from there. version.build() answers a different question, which commit is actually running: DPF_BUILD first, then a BUILD file the deploy writes, then git rev-parse --short HEAD when the checkout still has .git. The two can legitimately disagree — a box a commit behind the tag it meant to run shows the same version and a different build. See Deploy for the steps that keep the two in step.

The limits of this shape

Two gunicorn workers write to the same SQLite file. WAL mode and a 30-second busy_timeout (dpf/db.py) let the second queue for the lock rather than fail with database is locked, but queueing is not concurrency, and a third worker would only queue more often, not raise a ceiling that is not there. The warehouse cannot absorb that write path either: DuckDB's own multi-process story is "don't" (dpf/warehouse.py). One box, one gunicorn unit, two workers queuing behind one writer is the shape that SQLite's and DuckDB's own write stories hold the design to today — see Quality for the consequence of that ceiling and what would retire it.