We had a telemetry platform where reads and writes wanted opposite things.
Writes wanted to be small, constant and never blocked — devices in the field do not wait, and backpressure on ingest means data loss. Reads wanted wide aggregations across everything that had ever arrived, because that is what a fleet engineer looking at a diagnostic dashboard needs. Making one schema serve both meant making both bad: indexes that helped queries slowed ingest, and normalising for write efficiency made every read a seven-table join.
Command Query Responsibility Segregation fixed it. I would make the same call again. I also want to write down what it cost, because the pattern is usually presented with only one column of the ledger filled in.
The shape of the fix
Separate the models entirely. Writes land in a store optimised for ingest. A projection consumes the write stream and maintains a read model shaped like the questions people actually ask. Queries never touch the write path.
// The projection is the whole design; everything else is plumbing.
// Note the error handling: a projection failure must be visible, because
// silent projection lag is the failure mode that erodes trust in the data.
@EventHandler
public Mono<Void> on(TelemetryReceived event) {
return readModel
.upsert(DeviceSnapshot.from(event))
.doOnError(e -> log.warn("projection failed for {}", event.deviceId(), e))
.then();
}
Under peak load, queries went from multiple seconds to consistently sub-second. Ingest stopped being affected by dashboard traffic at all, which had previously been a genuine operational hazard — somebody opening a heavy report could degrade data collection.
The bill
Here is what does not appear on the slide.
| Cost | Where it lands |
|---|---|
| Eventual consistency | Every screen that reads after a write |
| Two schemas | Every migration, forever |
| Projection lag | A new incident class, and a new dashboard to watch it |
| Rebuild time | The first occasion you change the read model |
| Cognitive load | Every engineer who joins after you |
Eventual consistency is the one users notice. A device sends a configuration change, the user refreshes, and the old value is still there for a second. Technically correct, experientially broken. We handled it by having the write path return the expected post-write state so the UI could render optimistically — which works, and adds a second source of truth for the UI to be wrong about.
Rebuild time is the one that will actually hurt you. The first time you change the read model's shape, you must replay history to rebuild the projection. Over years of telemetry that is a long-running operation with its own failure modes, and it is the operation nobody has practised. Practise it deliberately, on production-scale data, before you need it. A projection you cannot rebuild is a projection you cannot evolve.
Cognitive load is the cost that compounds. Every new engineer must learn that there are two models, that one lags the other, and which one to trust for which question. That is a permanent tax on the team, paid in onboarding time and in the specific class of bug where somebody reads from the wrong side.
When it is worth it
Three questions, and I would want three yeses:
- Do reads and writes genuinely want different shapes? Or is the schema simply untuned? Add the index first. I have seen CQRS proposed for problems a covering index solved in an afternoon.
- Can the product tolerate a read that is a second stale? If any workflow requires read-your-write consistency, you are building a special case around your own architecture.
- Is there someone who will own the projection at three in the morning? Projection lag is an operational concern that needs a human with a runbook.
Two yeses out of three and you should tune the database. One yes and someone is pattern-matching on a conference talk.
What I would do differently
Build the rebuild path on day one. We wrote it when we first needed it, which is the worst possible time — under pressure, against a deadline, with no prior confidence it worked.
Make projection lag a first-class, visible number. Not a metric someone finds during an incident. A number on the main dashboard, with an alert, expressed in seconds behind. When users report "the data looks wrong", the first question is always whether the projection is caught up, and having that answer instantly is worth the effort of surfacing it.
Be far more explicit in the API about which side you are reading. We had endpoints that read from the read model and endpoints that read through to the write model for consistency-sensitive cases, and the naming did not distinguish them. That ambiguity caused real bugs. Name the seam.
Resist CQRS for the parts that did not need it. We applied it more broadly than the problem warranted, because once the machinery exists it is easy to reach for. Several aggregates would have been perfectly served by a single tuned table, and they inherited the whole tax for no benefit.
The general lesson
The pattern was right and my reasoning about it was too shallow. I evaluated whether CQRS would solve the performance problem — it obviously would — rather than whether the total cost over five years beat the alternatives.
That is the failure mode with any well-known pattern. It arrives with its benefits pre-argued by people who wrote about it, and with its costs distributed across a future team's time. The discipline worth building is asking what it costs in the third year, when the people who chose it have moved on and someone else has to change the read model.