Someone on the backend team renamed order_total to order_amount. Clean name. Makes total sense for their domain model. They shipped it on a Thursday afternoon. By Friday morning, your revenue dashboard was showing zero. Not wrong numbers. Zero. Because your Snowflake pipeline was still selecting order_total from the events table, and the column simply wasn’t there anymore.
You found out from a Slack message. From a director. At 9 AM.
This is the most common production incident in data engineering in 2026, and it’s almost never caused by bad code. It’s caused by the absence of a formal agreement between the team producing data and the team consuming it. That agreement has a name: a data contract. And most data teams still don’t have one.
The excuse is usually some version of “we move too fast.” The reality is that the teams who move fastest are the ones with contracts, because they stop discovering breaking changes from directors on Friday mornings and start catching them in CI on Thursday afternoons, before anything ships.
TL;DR
→ A data contract is a formal specification — schema, semantics, SLAs, ownership — between a data producer and its consumers. Not documentation. Enforcement.
→ Most data incidents don’t start with missing data or broken code. They start with a well-intentioned upstream change that silently invalidated an assumption someone downstream was relying on.
→ Contracts have three parts: schema (structure and types), semantics (what fields actually mean), and SLAs (freshness, completeness, availability). Schema-only contracts miss most real breakages.
→ The dual-write pattern is the only safe migration path for breaking changes: keep old field + add new field → both populated during transition → deprecation notice with a hard date → removal at v2. Each phase takes at minimum 30 days. Skipping phases causes incidents.
→ 90 days minimum notice for breaking changes. Data pipelines have long release cycles; consumers need time to update downstream logic, tests, and dashboards.
→ A contract not enforced in CI is just documentation. The ODCS (Open Data Contract Standard) YAML spec plus `datacontract-cli` gives you executable, version-controlled contracts in about 30 minutes per dataset.
→ dbt integration: map contract checks to dbt tests. Require a version bump plus consumer sign-off on breaking changes before merge. After one month of this, most teams report significantly fewer schema surprises.
→ The worst gotcha: contracts that only cover schema, not semantics. A field that changes meaning without changing type is undetectable to automated checks — and it’s how revenue figures silently drift for weeks.
Why schemas break and who owns the blame
Schema evolution sits between two teams that don’t talk to each other on the same cadence. The producer team — usually a backend or platform engineering team — is shipping product features, often weekly, and treats every field they emit as their own. The consumer team — your data engineering team — is running pipelines that depend on those fields staying stable, and finds out about breaking changes the same way archaeologists find ruins: by digging through wreckage.
The producer isn’t wrong for evolving their schema. The consumer isn’t wrong for depending on it. The incident happens because there was no shared definition of what “a safe change” means, no process for communicating it, and no tooling to enforce the agreement. The blame falls on the process, not the person. Which means the fix is a process change, not a person change.
Schema evolution is the load-bearing problem in data engineering in 2026, and it’s the problem most teams handle the worst. The good teams treat upstream schemas as contracts and run checks against those contracts on every pipeline run. The teams that lose stakeholder trust treat upstream schemas as suggestions and find out about every breaking change from a Slack message that starts “hey, the dashboard looks weird.”
That Slack message is always sent on a Friday. It is always sent to a director.
What a data contract actually contains
The mistake most teams make when they start with data contracts is writing schema-only contracts. Field names, data types, nullability. It feels rigorous. It catches a specific class of errors — column removed, type changed — but misses most real incidents.
Real breakages happen at the semantics layer. The producer changes order_total from gross to net revenue. Same field name. Same FLOAT type. No schema violation. But your revenue dashboard is now off by 23%, silently, because the number means something different than it did last week. A schema validator cannot catch this. Only a semantic contract can — one that documents what a field means, how it should be used, and what constitutes a valid business interpretation of its values.
A complete data contract has three layers. Schema: field names, data types, nullability, constraints (no negative values in a price field, for example). Semantics: what each field means in business terms, how it maps to domain concepts, what transformations are applied before it reaches the consumer. SLAs: freshness guarantees (this dataset is refreshed within 15 minutes of source update), completeness thresholds (at least 99.5% of expected rows must be present), availability targets, and a named owner with actual contact information — not “data team.”
The Open Data Contract Standard and the YAML spec
The good news for teams starting in 2026 is that there’s a growing standard: ODCS (Open Data Contract Standard), a YAML-based specification that defines schema, quality rules, SLAs, and ownership in a single document. It’s human-readable, version-controllable in git, and machine-parseable by tools like `datacontract-cli`, which can validate contracts, run compatibility checks, and generate reports.
A minimal ODCS contract for an orders dataset looks like:
dataContractSpecification: 0.9.3
id: orders-v1
info:
title: Orders
version: 1.0.0
owner: [email protected]
servers:
production:
type: snowflake
database: PROD_DB
schema: PUBLIC
table: orders
models:
orders:
fields:
order_id:
type: string
required: true
description: Unique identifier for the order
order_amount:
type: number
required: true
description: Net revenue after discounts and returns, in USD
minimum: 0
created_at:
type: timestamp
required: true
servicelevels:
freshness:
description: Data refreshed within 15 minutes of source update
threshold: PT15M
completeness:
description: At least 99.5% of expected rows present
threshold: "99.5%"This is not documentation theater. This YAML file is executable. `datacontract-cli test` validates your actual Snowflake table against this contract. It checks types, required fields, minimum values, and can be wired into CI so that any schema change that would violate the contract fails the PR before it merges.
The only safe migration path for breaking changes
When a producer needs to make a breaking change — remove a field, rename it, change its type, change its semantics — the contract provides a coordination mechanism. There’s a specific pattern that works, and teams that skip steps in it pay for it.
Day 0: Announce. The producer creates a deprecation notice in the contract YAML, updates the changelog, and notifies consumers via a designated channel. Critically, this notification includes a hard date for removal — not “eventually” or “when everyone has migrated.” Deprecated without a date is just a polite rumor. A field can sit in limbo for eighteen months while producers assume nobody uses it and consumers assume it will live forever.
Days 0–60: Dual-write. The producer populates both the old field and the new field simultaneously. Consumers can migrate on their own schedule during this window. The producer monitors usage of the old field (this is easy with Snowflake’s QUERY_HISTORY and column-level access tracking) to know when all consumers have switched.
Day 60: Deprecation notice with hard date. Consumers who haven’t migrated get a 30-day final warning. This is the reminder that actually motivates stragglers. The hard date is non-negotiable.
Day 90+: Removal at v2. The old field is gone. The contract version bumps to 2.0.0. This is a semantic major version — it breaks backward compatibility — and that bump is what triggers automated alerts to any consumer still on v1.
No drama. No guessing. No 2 AM rollback. Give consumers at least 90 days notice for breaking changes. This seems long, but data pipelines have long release cycles, and consumers need time to update downstream logic, tests, and dashboards.
Making it executable: CI enforcement that actually works
The critical architectural decision with data contracts is this: a contract not enforced in CI is just documentation, and documentation drifts. Within six months, the contract YAML and the actual schema diverge, nobody updates the contract when they ship features, and you’re back to tribal knowledge with extra steps.
The enforcement pattern that works:
1. Compatibility check on PR. Before any schema change merges, run `datacontract-cli diff` against the current production contract. Breaking changes fail the PR automatically. Non-breaking changes (adding a nullable field, loosening a constraint) pass. The definition of “breaking” is explicit in the contract spec, not up to whoever reviews the PR.
2. Consumer sign-off for breaking changes. If a breaking change is intentional (the producer knows and has planned for it), the PR requires explicit approval from all registered consumers of that dataset. This is enforced via GitHub CODEOWNERS or equivalent. Producers can’t ship breaking changes unilaterally.
3. dbt test integration. Map contract quality rules to dbt tests. Freshness SLAs become `dbt source freshness` checks. Completeness thresholds become row count assertions. Not-null requirements become `not_null` tests. These run on every dbt build, so violations are caught before models complete — not after reports are wrong.
4. Runtime validation at ingestion. Before data loads into your Silver or Gold layers, validate incoming records against the contract. Rows that violate constraints get quarantined in a dead-letter queue, not silently loaded as nulls. This catches semantic drift that schema validation misses: an order_amount field that’s suddenly returning negative values because someone upstream changed the sign convention.
The gotchas that sink most implementations
Exposing raw transactional schemas as data products. This is the most common structural mistake. When your data contract directly mirrors your application’s OLTP schema, every application refactor becomes a consumer’s problem. The fix is a stable abstraction layer — expose only what consumers need, not the underlying operational detail. Schema changes to the application layer should be absorbed by your ingestion layer, not propagated downstream.
Brittle contracts that break more than they prevent. Strict attribute lengths, tightly constrained enums, or hyper-specific format requirements feel like good quality controls. In practice, they make schemas so rigid that producers constantly need change approvals for minor operational updates that have no downstream impact. Design contracts around semantic guarantees and business invariants, not implementation details. amount > 0 is a semantic guarantee. DECIMAL(18,4) is an implementation detail that will change.
Unclear ownership is the silent killer. Data contracts fail most often not because of tooling gaps, but because accountability is unclear. When something breaks, teams scramble to diagnose issues that fall between ownership boundaries. Every contract needs a named owner with actual incident-response obligations. Not a team. Not a Slack channel. A person whose name is in the contract and who gets paged when a contract violation is detected at runtime.
Semantic changes that look like no-ops. Changing what a field means without changing its name, type, or schema is the hardest class of breakage to catch. order_amount switching from gross to net. A user_id changing from internal to external identifiers. These require semantic versioning (a major version bump) and human review, not just automated compatibility checks. Your CI can catch structural breakage; only your team can catch semantic breakage.
Contracts that cover batch but ignore streaming. If you have a Kafka-based event pipeline feeding your Snowflake tables, the schema contract lives in the Kafka topic, not in the table. Changes to the Kafka Avro schema — registered in Confluent Schema Registry or AWS Glue — need the same versioning and deprecation discipline as your warehouse schemas. Most teams only contract the warehouse side and get burned by streaming schema changes that propagate silently into their pipeline.
The real cost math
Data engineering incidents from schema breakage are expensive in ways that don’t show up on warehouse bills. A typical schema incident at a mid-sized company looks like: 3–4 hours of two engineers debugging, 1 hour of a data analyst investigating wrong numbers, a director review, and a post-mortem. Call that 10 person-hours, at a blended rate of $150/hour. That’s $1,500 per incident.
Teams that experience two schema incidents a month — which is conservative for a team without contracts — are burning $3,000/month, or $36,000/year, on incidents alone. That doesn’t count the cost of wrong decisions made from bad data before the incident was even discovered. One revenue calculation running off a silent semantic change for three weeks is often worth more than a year of incident cost.
The tooling investment for data contracts — `datacontract-cli`, ODCS YAML per dataset, CI integration — is a few days of engineering time. The 90-day discipline is a process change, not a tooling cost. The math is not close.
Where to start (not where everyone starts)
Everyone says “start with your most critical datasets.” That’s correct but useless. More specifically: identify the three datasets that caused production incidents in the last 90 days. Start with those. Not your biggest datasets. Not your most complex. The ones that already broke something.
For each: write the ODCS YAML (schema + semantics + SLAs + owner). Add `datacontract-cli` compatibility checks to the PR workflow for that dataset. Map the quality rules to dbt tests. That’s the first sprint. After one month of this on three datasets, you’ll have a template, a workflow, and enough muscle memory to expand to the rest of the catalog without it feeling like a governance initiative nobody asked for.
The one principle
Change is inevitable. Unmanaged change is expensive. A data contract is the agreement that makes change boring instead of dangerous. The goal isn’t to prevent schemas from evolving — schemas should evolve as the business evolves. The goal is to make every evolution visible, deliberate, and announced far enough in advance that nobody finds out about it from a director on a Friday morning.
Related reading: Open Data Contract Standard (ODCS) · datacontract-cli on GitHub · dbt State: Skip Unchanged Nodes, Cut Runtime by 60% · dbt Fusion: 30x Faster Parsing · Snowflake Query Execution: What Really Happens