Platform Teams Own the Pipeline. Nobody Owns the Data

Here is a test you can run on your own organisation this afternoon. Pick a dataset that something important depends on: A revenue table, a customer view, a feed a downstream team pulls from. Now find out who owns it.

You will find quickly who owns the cluster, the pipeline, the storage and the on-call rota if the job fails. What you will struggle to find is a person whose job it is to say whether the numbers in it are right.

That gap is not an accident. It is the direct result of how we drew ownership lines when platform engineering matured.

We Defined Ownership Around the Runtime

Platform engineering solved a real problem, and it solved it for a particular kind of workload. Microservices sit inside bounded contexts. They expose deterministic interfaces. They answer liveness and readiness probes. Ownership works there because the thing you own, the thing you measure and the thing that fails are all the same thing.

Data is stateful, and that breaks the symmetry. The pipeline is a service and gets owned like one. The payload is an output, and outputs are not on anyone’s rota.

The divergence is worth naming precisely. A service fails loudly and syntactically. The process dies, the probe fails, the deployment rolls back. A data system fails quietly and semantically. Every container is healthy, every run is green, every readiness check passes, and the payload carries silent corruption: A currency mismatch, a join that dropped rows without complaint, a column that now means something slightly different from what it meant last month. Syntactic health tells you the machinery ran. It tells you nothing about semantic validity.

You can see it in the alerts. A failed job routes to a named team in seconds. A message saying a number looks wrong goes to a shared channel, gets three shrugs, and lands on whoever answered first.

Three Symptoms Worth Recognising

The first is that nobody can say what a dataset promises. Consumers infer it from what is there today, build against that, and are surprised when it changes. No stated contract means every change is potentially breaking and none is officially breaking.

The second is that schema changes ship without consultation, and the contrast with service development is stark. In microservices, shipping an unversioned breaking payload violates semantic versioning and fails CI before it reaches a consumer. In data, an engineer alters a column type or drops an attribute directly in production, because no schema contract is enforced at the pull request level. Nothing in the path is capable of failing. You find out when someone downstream breaks.

The third is that nothing gets retired. Every organisation I have worked with produces datasets that cost money and are read by nobody. They persist because deletion requires knowing who depends on them, and that requires an owner.

Why the Usual Fixes Do Not Hold

Two responses are common and both under-deliver.

• A catalogue records what exists rather than creating accountability. An entry with a stale owner field documents the problem instead of solving it.
• A central data team that owns everything works at small scale, then becomes the bottleneck it was meant to remove. It also puts ownership furthest from the domain knowledge needed to exercise it.

Cloud-native architecture rejected both patterns for services years ago. The same reasoning has mostly not been applied to data.

What Ownership Actually Requires

Ownership is not a name in a spreadsheet. For a service, we expect four things, and all four transfer cleanly.

1. A named owner on the rota, not a team mailbox. If a data quality alert cannot page someone, it is not owned.
2. A stated interface: what fields exist, what they mean, what guarantees hold. You cannot version what you have not defined.
3. Known consumers. Services get this free through call logs. Data needs it built deliberately, through registration or query attribution.
4. A deprecation path, with a migration window and a real end date. Its absence is why nothing gets deleted.

Data Contracts as Declarative Infrastructure

The instinct will be to write a policy. Do not. Policies about data ownership get circulated and ignored at the same rate as every other policy.

Treat the data contract as declarative infrastructure instead. Kubernetes does not ask politely for resource limits and health probes. It refuses the pod. An internal data platform needs the same posture: the CI/CD orchestrator should reject any pipeline pull request where the dataset resource definition lacks three things. A declarative schema specification. An alert escalation routing key. An explicit TTL and retention manifest.

Declare those and ownership stops being a rule about the thing and becomes a property of it. Data quality alerts route through the same on-call system as service alerts, because the routing key sits in the manifest. The consumer list becomes a byproduct of access, and deprecation a supported operation with a defined window rather than a negotiation.

We measured this afterwards. Median time to identify an accountable owner for a dataset fell from about three days of asking around to under an hour. Roughly a third of the datasets we catalogued had no living consumer, and we retired them within two quarters. That was spend nobody could see while nobody was accountable for it.

The Uncomfortable Question

Cloud-native platforms have become very good at answering who is responsible when a system misbehaves. They are still poor at answering who is responsible when a system behaves perfectly and produces the wrong answer.

If a number in one of your datasets is wrong right now, how long before someone notices, and whose name is on it?

For most teams, the honest answer is that a consumer will notice eventually, and nobody’s name is on it. That is not a data problem. It is a platform design decision, and it can be reversed the same way it was made.