Designing Good Data Architecture: Reading Fundamentals of Data Engineering, Ch. 3

The previous post gave the whole lifecycle (what data is doing); this chapter asks — how do you design the system carrying it to be “good”? The most counter-intuitive and most memorable line in the chapter: good architecture isn’t a fixed blueprint, it’s a decision process that “trades off to buy flexibility and reversibility”. There’s no best architecture, only trade-offs that are relatively good in this context.

First, define “architecture”

The book’s definition is worth copying: data architecture is “the design of systems to support the evolving data needs of an enterprise, achieved by flexible and reversible decisions reached through a careful evaluation of trade-offs”. Taken apart, the keywords are three: trade-offs, flexible, reversible.

And a pair of words that often get mixed up:

• Architecture = why: why the system is cut this way, why this trade-off.
• Engineering = how: turning architectural decisions into something that runs.

This whole chapter is about why, not how.

Nine principles of good architecture

Borrowing from the AWS / GCP well-architected frameworks, the book gives nine principles. Condensed into one table:

Principle	In one line
Choose common components wisely	Shared things (storage, permissions, monitoring) should be chosen so the whole company benefits
Plan for failure	Assume things will break; think about availability and recovery targets (RTO/RPO) first
Architect for scalability	Scale out horizontally, and scale back in when not needed
Architecture is leadership	An architect’s output isn’t just diagrams, it’s leading people to the right trade-offs
Always be architecting	Architecture is a verb, not a document delivered once
Build loosely coupled systems	Components can evolve and be swapped independently
Make reversible decisions	Keep decisions retractable wherever possible
Prioritize security	Least privilege, zero trust; security is foundation, not a plug-in
Embrace FinOps	Cloud cost is a variable to keep designing for

The two worth digging into most, and most able to change how you decide, are reversible decisions and loose coupling.

Core one: reversible decisions — two-way doors vs one-way doors

The book borrows Jeff Bezos’s metaphor: decisions are two kinds of door.

• Two-way door (reversible): if you go the wrong way you can walk back. → Decide fast; if it’s wrong, change it.
• One-way door (irreversible): once through, it’s hard to return. → Decide carefully; worth spending time to evaluate.

The book’s claim: good architecture designs decisions as two-way doors wherever possible. Because requirements will change, and keeping “the ability to change your mind” is value in itself. In practice this means “don’t lock yourself in” — don’t sign a five-year contract, don’t let data live only in a proprietary format, don’t let one choice bind the next ten.

Core two: loose coupling — components evolve on their own

In a tightly coupled system the components are tangled together; change one and the whole body moves. In a loosely coupled system components talk through clear interfaces and can be developed, deployed and swapped independently.

It’s the same spirit as Kafka decoupling producers from consumers — reduce the dependency between components to “one clear interface”, and the system can evolve locally instead of being rebuilt wholesale.

Brownfield vs greenfield

One last practical distinction: are you building on a brownfield or a greenfield?

• Brownfield: rebuilding on an existing system. Constrained by what’s there, technical debt, no downtime allowed — low freedom, but the risks are concrete.
• Greenfield: a blank sheet. High freedom, but easy to over-design and underestimate the unknowns.

The book’s reminder is on point: greenfield freedom is a double-edged sword; don’t mistake “no baggage” for “anything goes”.

Reflections

”No best architecture, only trade-offs” is the line to internalise from this chapter

The more I do this the more I agree: architectural maturity is the shift from “finding the optimum” to “articulating the trade-off”. Beginners ask “which architecture is best”; the experienced ask back “under your constraints, what are you willing to trade for what”. It’s the same nerve as my concept note confirm the pain first, then bring the heavy weapons — a “best practice” without context is empty talk; every choice should come back to “this pain, this scale, this team” to be weighed. The book makes that the first principle, and I’m sold.

Reversibility is the “insurance” I’ve been buying all along

The “make it a two-way door where you can” rule is something I’ve practised unconsciously in a lot of posts. In Medallion I insist that Bronze is immutable and replayable — that’s essentially preserving reversibility: the raw data is still there, so however downstream changes you can redo it. Avoiding proprietary lock-in, keeping the raw layer, not signing long contracts — all the same move: spend a little now to buy the right to change your mind later. Requirements will change, and reversibility is the premium you pay against “uncertain”.

”Architecture is leadership”, “always be architecting” — these pull architecture back from technology to people

My most unexpected takeaway was the book listing “architecture is leadership” and “always be architecting” as principles. It’s saying: architecture isn’t a diagram a senior engineer finishes behind a closed door; it’s an ongoing, social activity — you make trade-offs with the team, evolve with requirements, keep communicating and correcting. That connects with what I got from reading the Tech Leader series: a good technical decision is never just a technical question, it’s “how do you lead a group of people to keep making the right trade-offs under uncertainty”. Drawing the architecture on the whiteboard is only the start; making it live in the team’s shared understanding is the real skill.