Tech

user one domain one public IP GET shop.com/api Ingress L7: reads the HTTP Host + path shop.com/ shop.com/api img.shop.com terminates TLS here too (https → http) web-svc ClusterIP → web Pods api-svc ClusterIP → api Pods img-svc ClusterIP → img Pods one IP, one entrance, routed by URL to many internal Services — something an L4 Service can't do

Ingress and Cluster DNS: One Entrance In, One Name to Recognise Each Other

· tech · 13 min read · 📚 Kubernetes — Learning Notes #9

The fourth post gave short-lived Pods a fixed address, the Service, but left two loose ends: first, …

#kubernetes#networking

Pod nodeSelector / affinity "I want a disk=ssd node" toleration: immune to gpu taint Node label: disk=ssd taint: gpu=true:NoSchedule non-immune Pods are driven off pull: Pod picks a node by label push: node drives off non-immune Pods a toleration is only immunity; it doesn't "attract" the Pod

Advanced Scheduling: Getting Pods onto the Right Node

· tech · 20 min read · 📚 Kubernetes — Learning Notes #7

The second post covered how the Scheduler picks a node for a Pending Pod in two steps: filter first …

#kubernetes#scheduling

PV / PVC: separating "demand" from "supply" StorageClass: dynamic provisioning — a PVC arrives, a PV is created Podwants a disk PVC (demand)"I want 10Gi · RWO"all the app developer asks for bound PV (supply)the actual 10Gi of storagebackend: EBS / NFS / Ceph… Supply / demand separation: the app says "how big, which access mode", never what hardware access modes: RWO (one node) · ROX (many nodes, read-only) · RWX (many nodes read-write; needs NFS or similar) reclaim policy: after the PVC is deleted, Retain the PV or Delete it (backend included)

K8s Storage: Volumes, PV/PVC and StatefulSets

· tech · 13 min read · 📚 Kubernetes — Learning Notes #6

Pods are short-lived and disposable — self-healing swaps them out at any moment. But some things mus…

#kubernetes#storage

Three breaches, three different shapes Penetration query a "nonexistent" key not in cache, not in DB → cache can never block it every one hits the DB Breakdown one hot key ⏰ just expired a burst of concurrent misses → all rush to rebuild from DB concentrated on one point Avalanche many keys ⏰ expire together (or Redis goes down entirely) → misses across the board DB collapses → cascade in common: requests bypass the cache and flatten the DB; the difference is the breach — a nonexistent key (penetration) · one hot spot (breakdown) · a whole swath (avalanche)

The Three Cache Disasters: Penetration, Breakdown, Avalanche, and the Right Fixes

· tech · 11 min read · 📚 Redis — Learning Notes #6

Using Redis as a cache, the classic pattern is cache-aside: a read checks the cache first, returns o…

#redis#cache

One image, every environment imagemy-app:1.0(immutable) DevelopmentConfigMap + Secret StagingConfigMap + Secret ProductionConfigMap + Secret Pod (dev) Pod (staging) Pod (production) 12-factor: config in the environment, never baked in — only then can one image be reused

ConfigMap and Secret: Pulling Configuration and Secrets Out of the Image

· tech · 12 min read · 📚 Kubernetes — Learning Notes #5

Over the previous posts you've learned to deploy an app and expose it as a service. But a real app i…

#kubernetes#concept

How expired keys get cleared: lazy + periodic, two prongs key whose TTL has expiredstill occupies memory (not cleared yet) ① lazy deletion (passive)someone GETs it →found expired → deleted on the spot, returns nil ② periodic deletion (active)background samples ~10×/s →random batch from keys with TTL, delete expired so "expired" ≠ "freed" — untouched and not yet sampled, it just lies there

Redis Expiration and Eviction: TTL, Lazy Deletion and maxmemory Policies

· tech · 12 min read · 📚 Redis — Learning Notes #5

Use Redis as a cache and sooner or later you hit two questions: how does a key with a TTL get cleare…

#redis#cache

Snapshot (RDB) vs running log (AOF) RDB snapshotperiodic full picture loss window ⚡ small file, fast load, good for backups ✓ | lose what happened between snapshots (maybe minutes) ✗ AOF logevery write logged ⚡ safer (lose at most one fsync interval) ✓ | big file, slow load (replay) ✗ Hybrid (Redis 4+): RDB snapshot at the head of the AOF + incremental commands → fast load, little loss (recommended)

Redis Persistence: RDB Snapshots vs AOF Logs, and Whether Data Is Actually Lost

· tech · 14 min read · 📚 Redis — Learning Notes #4

"Redis is an in-memory database; lose power and all the data's gone" — that line is half right, half…

#redis#persistence

Single-machine cron is easy; "reliable" cron is hard cron (one machine)time's up, run it — trivial ✗ machine dies → all schedules stop (SPOF) must runeven if it dies replica (leader) replica / standby replica / standby consensus log (Paxos)records: which jobs have run one dies → re-elect leader, restore state from consensus → no re-run, no skip Making the simplest cron reliable puts distributed consensus underneath it

Reliable cron: The Simplest Scheduled Job Gets Hard the Moment It's Distributed

· tech · 12 min read · 📚 Google SRE — Reading Notes #15

cron may be the simplest piece of infrastructure there is: time's up, run a job. Anyone who has writ…

#sre#reliability

Single thread + event loop: one queue, one at a time client client client client 10k+ conns epollI/O multiplexingone thread watches all command queue (one line)cmd1cmd2cmd3 single-thread executionone at a time, each atomicno locks, no races Redis 6's "multithreading" is only for socket reads/writes, the network chores — command execution stays single-threaded, so atomicity and lock-freedom are unchanged

Why Is Single-Threaded Redis So Fast? And the Landmine of O(N) Commands

· tech · 11 min read · 📚 Redis — Learning Notes #3

The first post said one reason Redis is fast is "single thread + no locks". That sounds backwards — …

#redis#performance

Five core structures: pick the right one, the problem is half solved Stringbytes / number 42 cache · counter (INCR) · distributed lock (SET NX) Listordered, both ends queue (LPUSH / RPOP) · latest N items (LPUSH+LTRIM) Hashfield → value name: Aidanage: 30 store objects · change one field without moving the whole thing Setunordered, deduplicated dedup · tags · intersection (mutual friends, SINTER) Sorted Setscored members, kept sorted a:1b:2c:3 leaderboard · range queries · delayed queue (score = time)

The Soul of Redis: Five Core Data Structures + Advanced Weapons

· tech · 12 min read · 📚 Redis — Learning Notes #2

The previous post said the soul of Redis is "data structures" — so this one opens the toolbox. Ninet…

#redis

Traditional KV cache (memcached) key → "a blob of string" (opaque) changing one field = GET whole → edit in app → SET whole Redis (data structure server) key → List / Hash / Set / ZSet operate server-side (atomic) = HSET · LPUSH · ZADD · ZRANGE e.g. leaderboard = one Sorted Set, ZADD to score + ZRANGE for top N no fetching the whole board to sort in the app — compute next to the data, fast and atomic

What Is Redis? Not Just a Cache, but an In-Memory Data Structure Server

· tech · 14 min read · 📚 Redis — Learning Notes #1

Most people first meet Redis as a "cache" — throw database query results in, grab them next time. Th…

#redis#concept

Grab one real request and follow it all the way userone real request entry LB(in-house) API service(in-house framework) queue(in-house) databasefinal stop Stop at every hop and ask these four ① what is this? who maintains it? ② how do I see its health? monitoring / logs? ③ if it breaks, what happens downstream? ④ normally, what do traffic / data look like? One lap and you hold a "living" architecture map — more accurate than any wiki

An SRE Parachuted into a 'Build Everything In-House' Company: Standing Firm in the First 90 Days

· tech · 22 min read · 📚 Google SRE — Reading Notes #14.5

The most thrilling kind of job change is joining a company that builds almost everything itself: no …

#sre#career

Homebrew leader election: the network cuts, each side rules itself network cut ✂ Left half: Node A "no word from B → I'm leader" write arrives: X = 1 Right half: Node B "no word from A → I'm leader" write arrives: X = 2 Network heals: is X 1 or 2? both think they're right → data diverges, unrecoverable (split brain)

Distributed Consensus: How Machines That Crash Agree on One Thing

· tech · 19 min read · 📚 Google SRE — Reading Notes #14

A group of machines agreeing on "one thing" — who is the leader? who holds this lock? what's the lat…

#sre#distributed-systems

One request, two layers of load balancing useranywhere ① Frontend LBwhich datacenter?DNS · Anycast · VIP by geography / health / capacity datacenter(chosen) ② In-datacenter LBwhich machine?by real load / health task 1 task 2 task 3 handles "across datacenters":proximity, avoid dead sites, spread capacity handles "inside the datacenter":don't overload one, don't feed the broken

Load Balancing: Pick the Right Datacenter, Then the Right Machine

· tech · 11 min read · 📚 Google SRE — Reading Notes #13

Between a user sending a request and the request being handled, it passes through two layers of load…

#sre#networking

Automation's evolution: climb up, take the human out of the loop automation ↑ · fewer hands ④ Autonomous / self-healingthe system runs itself → humans leave the loop ③ General automation platformreused across systems, consistent, scalable ② Task-specific scriptssave time, but need care; new scenario, they break ① Manual operation (toil)slow, error-prone, inconsistent, can't scale ⚠ But automation amplifies the blast radius do it right consistently, do it wrong consistently — one bad click can switch off a whole datacenter

Automation, Release Engineering and Simplicity: Making Change Fast and Safe

· tech · 15 min read · 📚 Google SRE — Reading Notes #12

Automation, release engineering, simplicity — at first glance three unrelated topics. Put them side …

#sre#automation

what you think is safe backups every day ✓✓✓ safe "on paper" when you actually restore… it falls apart for real backup file itself corrupt (never verified) restore procedure never run → scramble restore too slow → SLA blown, data gone never-restored backup = Schrödinger's backup (alive or dead? open it to find out) what matters isn't the "backup", it's the "recovery" — the backup is only the means

Data Pipelines and Data Integrity: Having a Backup Doesn't Mean You Can Restore

· tech · 10 min read · 📚 Google SRE — Reading Notes #11

Beyond the "service stays up" reliability covered so far, a data system has a more fundamental layer…

#sre#data-engineering

one falls → shifted traffic crushes the next → domino wipeout Server Aoverloads first → down ✗ Server Btakes A's traffic → down ✗ Server Cgets everything → down ✗ and the retry storm pours fuel on the fire: requests fail / slow clients retry total load rises positive feedback: more failure → more retries → more failure (self-amplifying)

Cascading Failures and Overload: Don't Let One Server Take Down the Rest

· tech · 9 min read · 📚 Google SRE — Reading Notes #10

The first post said the goal of reliability is "keeps working when things go wrong". But one kind of…

#sre#reliability

E2E Integration Unit ↑ few, slow, brittle (full user path) components assembled (service + DB…) ↓ many, fast, stable (one function) Inverted (lots of E2E, few unit) = anti-pattern: slow, flaky, and hard to tell which layer broke

Testing for Reliability: Tests Don't Prove the Absence of Bugs, They Let You Move Fast

· tech · 10 min read · 📚 Google SRE — Reading Notes #9

This post is about something usually filed under "the developers' business" that is in fact a corner…

#sre#reliability

interface DevOps ← philosophy / culture: defines "what to do" reduce silos · failure is normal · incremental change · measure everything · automate implements (gives the concrete how) class SRE implements DevOps ← the how error budget · blameless postmortem · canary · SLI/SLO · eliminate toil DevOps is direction and principles, SRE is one concrete implementation — not either/or, different layers

DevOps vs SRE: One Is an Interface, the Other an Implementation

· tech · 8 min read · 📚 Google SRE — Reading Notes #1.5

With the previous post covering what SRE is, let me deal with a comparison that gets asked constantl…

#sre#culture