Tech

Blame culture · vicious cycle incident "whose fault?" hunt the culprit people hide mistakes, stay quiet nothing learned → repeat Blameless · virtuous cycle incident "why did the system allow it?" everyone tells the whole story fix the system → steadily stabler Same outage; asking "who" or asking "the system" takes the team down opposite cycles

Blameless Postmortems: Turning Outages into Organisational Learning

· tech · 11 min read · 📚 Google SRE — Reading Notes #7

The previous post was about finding the root cause — but then what? The postmortem turns one expensi…

#sre#culture

① Triage keep the system alive (root cause can wait) ② Examine monitoring / logs four golden signals ③ Diagnose hypothesise → test → eliminate bisect to close in ④ Treat one variable at a time reversibly not fixed? new hypothesis Every step "narrows the range"; the anti-patterns (guessing, swapping parts, changing many things) never narrow, only gamble

Effective Troubleshooting: Debugging Is a Method, Not a Talent

· tech · 10 min read · 📚 Google SRE — Reading Notes #6

The previous post said on-call stops the bleeding first; but once the bleeding has stopped, you stil…

#sre#incident

An alert comes in: does it need a person? How urgently? Page a person must intervene "now", or users are being affected → wake someone Ticket a person needs to handle it, but it isn't urgent → working hours Log nobody needs to look; keep it for reference / later analysis → disturb nobody Stuff the non-urgent into Page → alert fatigue, crying wolf; the real incident gets ignored

Alerting and On-Call: When to Wake Someone Up

· tech · 11 min read · 📚 Google SRE — Reading Notes #5

The previous post ended on a line: when should you wake someone up? This post answers it. It's reall…

#sre#incident

Relational users orders FK tables + foreign keys, SQL many-to-many, strong joins Document name, email positions [ … ] (nested) education [ … ] (nested) nested JSON, natural one-to-many good locality (one read, whole thing) Graph vertices + edges, highly connected the relationships are the point

Data Models: Relational, Document, Graph — What Are You Actually Choosing?

· tech · 10 min read · 📚 Designing Data-Intensive Applications — Reading Notes #2

The first post covered what a data system should pursue (reliable, scalable, maintainable). This one…

#distributed-systems#book-notes#data-modeling

Coordinator receives, plans, dispatches Segment 1 Segment 2 Segment 3 DISTRIBUTED BY (key): hash(key) decides which segment each row lands on; queries run on every node in parallel ⚠ uneven key → one segment overloaded (skew), parallelism lost, the slowest node drags down all

When SQL Runs on MPP: Greenplum and Cloudberry

· tech · 13 min read · 📚 SQL: I Thought I Knew It #12

The finale of the whole SQL series. The previous 11 posts all ran on single-node PostgreSQL; this on…

#sql#data-engineering

① Dirty Read T1 T2 balance → 200 (uncommitted) reads balance 200 ❌ ROLLBACK ② Non-repeatable Read T1 T2 reads balance 100 balance → 200, commits reads balance 200 again ❌ ③ Phantom Read T1 T2 finds 3 orders inserts 1 matching order, commits now finds 4 orders ❌ time →

Transactions and Isolation Levels: Graded Ways to Keep Concurrency from Fighting

· tech · 19 min read · 📚 SQL: I Thought I Knew It #11

Everything so far has been "how one query runs fast" (indexes, EXPLAIN); this post switches dimensio…

#sql#concept

Nested Loop outer outer outer inner(faster if indexed) each outer row → one inner lookup suits: small tables / indexed inner Hash Join small hash table (memory) big small builds hash, big probes suits: big tables, equality joins Merge Join 1 3 5 2 4 6 sort both sides, merge like a zipper suits: already sorted / indexed

Reading EXPLAIN: How the Optimizer Actually Runs Your Query

· tech · 10 min read · 📚 SQL: I Thought I Knew It #10

The previous post left a question: is the index actually being used? The answer is in EXPLAIN. This …

#sql#performance

Reliability keeps working when things go wrong · fault-tolerant, not fault-free · fault ≠ failure · deliberately inject faults hardware / software / human error Scalability holds up as load grows · describe the load first · measure with percentiles · scale up vs scale out know the load before adding machines Maintainability lets people work on it well · Operability · Simplicity · Evolvability complexity is paid for by others Three non-functional requirements — features decide if it works; these decide if it survives

Reliable, Scalable, Maintainable: The Three Goals of a Data System

· tech · 14 min read · 📚 Designing Data-Intensive Applications — Reading Notes #1

Starting a new series: reading Martin Kleppmann's Designing Data-Intensive Applications (DDIA). It c…

#distributed-systems#book-notes

GROUP BY date_trunc directly datecount 7/013 7/025 no row for 7/03 ✗ 7/042 fill the hole generate_series + COALESCE 0 7/013 7/025 7/030 7/042 GROUP BY only emits buckets that "have data" — 7/03 with no orders vanishes, and the series breaks

Time Bucketing and SCD: Two Traps in Handling Time with SQL

· tech · 10 min read · 📚 SQL: I Thought I Knew It #8

Following the previous post's "ranges", this one covers two traps time sets in SQL — both rooted in …

#sql#data-engineering

your service ① Latency how long until a request returns? track success and failure latency separately ② Traffic how busy is the system right now? QPS / requests per second ③ Errors how many requests failed? failure rate (incl. "200 but the content is wrong") ④ Saturation how close to the limit? resource utilisation → the best early warning First three = user experience (usable directly as SLIs); the fourth, saturation = how long you can hold

Monitoring: The Four Golden Signals

· tech · 12 min read · 📚 Google SRE — Reading Notes #4

The previous post said the SLI is the measured reliability number — and those numbers come from moni…

#sre#monitoring

Is it toil? Check six traits ✓Manual — a person does it, step by step ✓Repetitive — done many times, and will be done again ✓Automatable — a machine could do it; nobody has written it yet ✓No enduring value — the system is no better afterwards ✓Scales linearly — the service grows, so does it ✓Reactive — done when triggered, not planned More matches → more toil. Note: meetings, planning, documentation are overhead, not toil

Eliminating Toil: Treat Repetitive Operations as Bugs to Be Killed

· tech · 12 min read · 📚 Google SRE — Reading Notes #3

The first post said SRE's core is "operations can be engineered". The toil in this post is the thing…

#sre#reliability

① Identical rows (1, A, 100) (1, A, 100) (1, A, 100) DISTINCT (1, A, 100) whole row identical → drop the extras DISTINCT / GROUP BY solves it ② Same key, several versions user1 · pending · 2/01 user1 · paid · 2/03 user1 · refunded · 2/05 ROW_NUMBER keeps latest user1 · refunded · 2/05 rows differ → DISTINCT does nothing you have to pick the "representative" row

Deduplication Done Right: DISTINCT Isn't the Only Answer

· tech · 11 min read · 📚 SQL: I Thought I Knew It #6

Duplicate data is almost daily life in data engineering: a pipeline rerun imports twice, CDC pulls i…

#sql#data-engineering

Input (orders): 4 rows A · 100 A · 250 B · 80 B · 120 GROUP BY (collapses) A · SUM 350 B · SUM 200 4 rows → 2 rows (one per group) SUM() OVER (no collapsing) A · 100group 350 A · 250group 350 B · 80group 200 B · 120group 200 4 rows → 4 rows (each gains a whole-group value)

Window Functions: Aggregation Without Collapsing

· tech · 11 min read · 📚 SQL: I Thought I Knew It #5

The previous post's GROUP BY collapses each group into one row. But you've surely met this need: "I …

#sql#concept

99.0% 100% SLA 99.5% external contract · breach pays out SLO 99.9% internal target (aim here) breach missed target (not yet a breach) healthy ← SLI 99.95% actually measured safety buffer: feel the pain inside before the customer does Strictness: SLA (loose) < SLO (strict) ≤ SLI (measured when healthy)

SLI / SLO / SLA: A Measurement, a Target, a Contract

· tech · 12 min read · 📚 Google SRE — Reading Notes #2

The previous post said error budget = 1 − SLO. But what is an SLO? And how does it differ from the o…

#sre#reliability

Set the reliability target at 99.9%, not 100% the remaining 0.1% isn't a regret, it's a "budget" you can spend Error Budget = 1 − SLO requests succeed ≥ 99.9% (SLO target) fail ≤ 0.1% (exaggerated) ≈ 43 min/month of downtime Chasing 100%: cost explodes, marginal benefit → 0 — and users can't tell 99.9% from 100% (their network, phone and Wi-Fi were never that reliable anyway)

What Is SRE? Start with the Error Budget

· tech · 13 min read · 📚 Google SRE — Reading Notes #1

"SRE" is a hot word, and it's very often misread as "slightly more advanced operations" or "a sysadm…

#sre#reliability

raw rows (orders) A · amount 100 A · amount 250 B · amount 80 B · amount 120 B · amount 50 GROUP BY customer collapsed: one row per group customer = ACOUNT=2 · SUM=350 customer = BCOUNT=3 · SUM=250 amount has several values per group (100/250) → can't select it bare; aggregate (SUM/AVG…) into one number

GROUP BY: Collapsing Many Rows into One

· tech · 12 min read · 📚 SQL: I Thought I Knew It #4

Everyone can write GROUP BY, but nearly everyone has also been stopped by column "..." must appear i…

#sql#concept

SQL logic has three values (not two) TRUE condition holds FALSE condition fails UNKNOWN don't know ↑ every comparison with NULL lands here age = NULL and age <> NULL alike WHERE / ON / HAVING let only TRUE through TRUE → ✓ kept FALSE → ✗ dropped UNKNOWN → ✗ dropped

NULL Isn't a Value, It's "Don't Know"

· tech · 13 min read · 📚 SQL: I Thought I Knew It #3

The previous post's LEFT JOIN pads NULL for rows that matched nothing. That NULL is this post's subj…

#sql#concept