Data engineer (data hub)
You build the data product, and you are accountable for it working — not for whether it should have been asked for. You design the model, write the data agreement (contract), and decide when a release is good enough to promote or to roll back.
What you decide
- The data model — tables, grain, keys, attributes — on Design the tables.
- Column descriptions, signed off once written: Sign off, on What it means.
- The agreement's text — Write it myself or run the agent, then Save my version. Accepting it is the use case team's.
- Build and validate a release, push its plan to a tracker, and push its checks to Databricks.
- Approve and promote a release, and roll back to an earlier one.
Where you work
- Design and What it means, under Refine
- the Data contract step, and its agreed text
- Build and validate, under Deliver, plus Push to ‹tracker› and
plan.json - Go live, the release's own page, and Releases for rollback
Step by step
1. Design the data model. Extend the shared layer by name where it exists, and design the product's own tables on top: a grain, a primary key, and, where it should reconcile against a source, a business key. Save the design. Nothing with an error-severity problem can build.
2. Column descriptions. Write what a column means if nobody has, then Sign off. Editing a description after sign-off un-signs it: the approval is on these exact words, not on the column in general.
3. Data contract. Write it myself opens a blank draft; Save my version needs the decision and the Explicitly not for field filled in. Accepting the result is the use case team's, not yours.
4. Build and validate. Needs an accepted agreement. Build ‹version› and run its checks compiles the model to SQL, runs it, and checks every acceptance criterion. Only error-severity failures block a release; warnings ship, recorded. A blocked release stays blocked until you fix the model and build again.
5. Go live. Approve #N records what you checked; Promote #N to production materialises the table and publishes the agreement in one step. Promotion is refused on the worst reason first: access refused by a source owner, a declined agreement, a simulated approval, an unanswered demand, a missing countersignature, then simply no recorded approval.
6. Roll back. Only a release that was once approved and once live can be rolled back to, from its own page: Roll back to this.
Push the build's work breakdown to Azure DevOps or Jira — see Connecting a delivery tracker — and push the agreement's checks into Databricks as CHECK constraints, a DLT module or Lakehouse Monitoring definitions — see Enforcing the agreement in Databricks.
Note
A rollback is recorded as a promotion, not undone as a mistake. Roll back to an earlier release and the newer one is superseded on the record, not deleted.
What the record keeps about you
Every design you save, every description you sign off, every draft of the agreement, every build with its checks and the exact SQL that ran, and every approval and promotion, each with your name and when. A release's own page keeps the approval comment, every check result and that SQL permanently — it is what an auditor reads, not your memory of the build. See Audit events.