NetworkPolicy and CNI: The Firewall Between Pods

Service and Ingress were about “how traffic finds a service”, but underneath sits a more basic, more easily ignored question: can Pods reach each other by default? The answer shocks many people — everything is open by default; any Pod can connect to any other Pod in the cluster. This post covers two things: the layer that gives Pods a network at all (CNI), and the firewall that turns “all open” into “allow-list” (NetworkPolicy).

K8s’s network model: a fully connected flat network by default

K8s makes only one hard demand of the network: every Pod has its own IP, and any two Pods can talk directly by IP, across nodes or not, with no NAT in between. That brings a security fact that’s often underestimated: there is no isolation by default. The frontend Pod can reach the database Pod, service A can reach service B’s internal port — anyone who knows the other’s IP (or Service name) gets through. Convenient, but it also means once one Pod is compromised, it can move laterally across the whole cluster.

CNI: the layer that actually gives Pods a network

The promise “every Pod has an IP and they can all reach each other” is something K8s does not implement itself — it outsources it to a plugin standard, CNI (Container Network Interface). Every time the kubelet creates a Pod, it calls the CNI plugin to assign an IP, attach a virtual NIC, set up routes, and only then is the Pod actually on the network. So when the CNI is missing or broken, the classic symptom is Pods stuck in ContainerCreating and the node showing NotReady — not a scheduling problem; nobody wired up the network at all.

Common plugins are Flannel (a simple overlay that gives you only “all open”), Calico and Cilium (the latter two also enforce NetworkPolicy on top of connectivity). It’s another of K8s’s “leave a blank for plugins” designs, with the same flavour as Ingress needing a Controller: the core defines the spec; capabilities come from plugins.

NetworkPolicy: turning “all open” into “allow-list”

To switch off the default openness, you use NetworkPolicy. Its mechanism has one key twist you must commit to memory: as soon as a Pod is selected by any NetworkPolicy, it flips, in that direction (ingress / egress), from “default allow” to “default deny”, and from then on only traffic the rules explicitly list gets through.

A few properties that trip people up: it’s namespaced (governs only its own namespace); allow-list only, no deny-list (you can only list “who is allowed”, never “who is blocked”); multiple policies add up as a union (rules only ever widen, they never veto each other). To get “everything in this namespace denied by default”, apply a policy with podSelector: {} (selects every Pod) and no ingress rules at all — everyone selected, nothing allowed, so everything is shut, and then you open allow-list entries one by one.

One policy has “two selectors” — don’t mix them up

The most disorienting part of NetworkPolicy is that it contains two selectors with different jobs — one picks “whom to protect”, the other “whom to allow”:

In YAML: shut everything first, then open one allow-list entry

The safe approach in practice is two layers: first give the whole namespace a default-deny (close the front door), then open allow-list entries one by one. default-deny is “select every Pod, but give no ingress rules”:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: default-deny-ingress }
spec:
  podSelector: {}                 # empty = select every Pod in this namespace
  policyTypes: [ Ingress ]        # Ingress type only, and no from entries listed → inbound fully shut

Then separately allow “api may reach db on 5432” — note the two selectors inside have different jobs:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: db-allow-api }
spec:
  podSelector:
    matchLabels: { app: db }      # ① whom this rule "protects": db
  policyTypes: [ Ingress ]
  ingress:
    - from:
        - podSelector:
            matchLabels: { app: api }   # ② whom to "allow" in: api
      ports:
        - { protocol: TCP, port: 5432 } # narrowed to a specific port

Stacked together, the effect is the right half of the first diagram: db, selected by a policy, flips to default deny; only Pods labelled app=api can reach 5432; everything else is blocked. To allow another namespace, swap from to a namespaceSelector; to allow an IP range outside the cluster, use ipBlock. Rules only ever add, never subtract, and take the union — to be stricter, stack a narrower one on top rather than trying to write a “deny”.

A big pit: NetworkPolicy only works with a CNI behind it

This is the sneakiest point, and it ties the post’s two protagonists together: the NetworkPolicy object itself is only rules; what actually “drops packets” is the CNI plugin. If your cluster uses a CNI that doesn’t support policy (plain Flannel, say), then you can kubectl apply a pile of NetworkPolicies — and they’ll be quietly ignored; not one packet gets blocked. No error, no warning; you think the database is locked down, and the door is wide open. It’s exactly the same pit as an Ingress doing nothing without a Controller: the object is a desired state; something actually running, and supporting it, has to execute it. Before applying policies, confirm your CNI (Calico / Cilium and the like) really enforces them.

Reflections

”All open by default” is something I wish I’d known earlier

Early on I had a dangerous illusion: that once things were in the cluster, each service running in its own namespace, they were naturally isolated from each other. Badly wrong. K8s is by default a fully connected flat network; the frontend Pod can connect straight to the database Pod’s internal port — not a single wall. What truly woke me up was imagining “what happens when an externally exposed Pod is compromised”: under the default openness, the moment an attacker lands, the door to lateral movement across the whole cluster is wide open. Since then I treat NetworkPolicy as table stakes for going live, especially for databases and internal APIs, things that “should only be reachable by a specific few services” — secure by default is never free; you have to build the walls yourself.

Remember “allow-list, per-Pod, flips the default” once and you won’t misconfigure again

Nearly every mine I’ve stepped on with NetworkPolicy came from not having its model straight: it can only write allow, never deny; it takes effect per Pod, and Pods not selected by any policy stay fully open; and once a Pod is selected, that direction flips to default deny. Those three together explain the two most common newbie ghost stories — “I only meant to block web, and now nobody can reach db” (selection means shut everything; they forgot to add the allow-list), and “I wrote a policy and nothing happened” (the Pod wasn’t selected at all, or the CNI doesn’t enforce). Getting the mental model right is a hundred times more useful than memorising YAML fields.

”Spec and plugin” again — K8s’s beauty and pain live here

Finishing this post, I’m more certain that K8s’s soul is a kind of restraint: almost none of the hard capabilities are done by K8s itself; it defines an interface and leaves it to plugins. Networking (CNI), storage (CSI), L7 routing (Ingress Controller) all follow the pattern. The upside is enormous flexibility — you can swap in Calico, Cilium, any conforming implementation; the price is that “K8s is installed” doesn’t mean “the capability is there”; you have to know exactly what your cluster has plugged in and whether it supports the feature you want. Whenever I take over a cluster now, my first batch of questions always includes: “Which CNI? Does it support NetworkPolicy?” — because the answer directly decides whether the isolation rules I write are a real wall, or a sheet of paper pasted onto thin air.