Docs / How-to guides / Enforce the agreement in Databricks
How-to guide For operators

Enforce the agreement in Databricks

Compile a data contract's checks into what Databricks can actually enforce, instead of leaving them to run only here, and optionally push them onto the table.

Before you start

  • A published data contract — checks compile from an accepted one; there is nothing to push before that.
  • A Databricks connection, if you intend to install or check rather than only read the generated artefact — see Connect Databricks.
  • To install: the connection's token needs USE CATALOG, USE SCHEMA and MODIFY, plus ownership of the table — Unity Catalog reserves ALTER TABLE for the owner.
  • The release.build permission.

Steps

Either from the product's Quality evidence tab, or the command line — both read the same agreement and agree on what it compiles to.

In the app: open the product, then Quality (/p/<slug>/quality). Switch between the three targets; the report and the generated artefact are pure functions of the agreement, so all three are shown whatever the warehouse is. Only Delta constraints has buttons: Install these on <table> applies the constraint set, and Check what the table enforces reads it back.

From the command line:

python -m dpf quality                              # constraints, with the report
python -m dpf quality --target dlt --on-error drop  # a DLT module for their repo
python -m dpf quality --target monitor              # custom metrics for the SDK
python -m dpf quality --push                        # apply the constraint set
python -m dpf quality --check                       # read back what is enforced

Note

Install and check run through the same Databricks connection the Connections page lists as a mock-up: tested as text, never yet run against a real workspace.

Every run prints the faithfulness report before the artefact: which checks the chosen target carries in full, which only partly, which not at all, and why. A customer who believes a check runs in their warehouse when it does not is worse off than one who knows it runs only here, so nothing is dropped quietly — a check no target can carry is named in red.

Check that it worked

Installing reports the constraint names actually applied; it is not transactional, so a set that fails halfway leaves the earlier ones installed. Checking reads the constraints back from the table's Delta properties and reports drift four ways: missing (never pushed, or dropped by hand), changed (edited in place), extra (from a superseded agreement) and unmanaged (somebody else's). Everything this application installs is prefixed dpf_.

Note

Expect the first install against a table with any history to be refused — ADD CONSTRAINT validates every row already there, and real tables usually hold one that breaks the rule.

Which target to choose

Delta constraints DLT expectations Lakehouse Monitoring
Enforced by the engine, on every write the DLT pipeline, and nothing else nobody — it observes
On a violation the whole write fails fail, drop the row, or record it a number moves
Carries severity no — one, and it is fatal yes n/a
Right for not_null, range, accepted_values any row rule, on DLT or Lakeflow null_rate, row_count, trends

What is not supported, and why

  • foreign_key cannot be pushed to any target. A CHECK cannot reference a second table, a DLT expectation is a predicate over one row, and a monitor aggregates over one table alone. Referential integrity stays a check this application runs.
  • unique and row_count cannot be constraints or DLT expectations — neither is visible from one row. Monitoring carries row_count exactly and unique only approximately.
  • null_rate is a proportion, so it belongs to monitoring and nowhere else.
  • A warn is never pushed as a CHECK constraint — a CHECK has one severity, and it is fatal to the write.
  • A threshold above zero is never pushed as a CHECK constraint either. DLT gets it instead, as a recording expect.
  • expression is emitted verbatim. It is authored SQL; Delta refuses a CHECK that aggregates, subqueries or names another table, and the ALTER is where that is found out.

If it does not work

Symptom Fix
"this Databricks connection is read-only, so it will not …" Expected on a SELECT-only token — see Connect Databricks.
ADD CONSTRAINT refused on the first install A row already breaks the rule. Fix the data or the check, then push again — drop-first makes a second push safe.
"there is nothing to push for --target dlt/monitor" Only constraints install from here; the DLT module and the monitor definition are files for the customer's own pipeline repository and SDK.
Documentation for version 0.2.0 Reviewed 2026-10-19 Covers dpf/domain/quality.py, templates/quality.html