Learning about Kubernetes Networking Policies

Jumpstarting your understanding of Kubernetes Networking Policies.

We are about to get hands-on with Kubernetes and network policies by walking through some common use cases. By the end of this post, you should have a basic understanding of how to implement networking policies in Kubernetes, and how to allow / block traffic within your cluster.

Overview

• Set up the testing environment using Minikube
• Deploy an Nginx container to the defaultnamespace
• Deploy a container to a new namespace to use as an interactive shell to connect with the Nginx container
• Dive into use cases and policies to showcase how it works

Tools I’m using for this exercise:

• Minikube
• VS Code for editing
• Windows 10 Pro
• cmder terminal

If you are using Mac or Linux, you should still be able to follow along.

Setup

By default, Kubernetes allows all ingress and egress traffic.

1. Ensure you have the ability to control a test Kubernetes cluster. I use Minikube so here’s my command to start up a cluster with Calico installed. You can use any networking interface that supports Network Policies.

minikube start --network-plugin=cni --cni=calico

If you have Calico installed, see if it’s running.

kubectl get pods -l k8s-app=calico-node -n kube-system

Proof that I actually run the commands

Deploy an Nginx container

We are going to deploy an Nginx pod to use as the target server for this demo.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-for-calico-demo
  namespace: default
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.20.0
        ports:
        - containerPort: 80

1. Apply the YAML to your cluster.

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/deployments/nginx.yaml

2. Let’s create a service.

kubectl expose deployment nginx-for-calico-demo

We now have something to test against.

Deploy a container to a new namespace to use as an interactive shell to connect with the Nginx container

Now that we have an accessible web server in the defaultnamespace, let’s create a pod with an interactive shell so we can access it from within the cluster.

1. Create a new namespace

kubectl create ns airwave-dev

2. Run an interactive shell in the airwave-dev namespace and once that shell appears, use wget to access the Nginx pod by its service name.

kubectl run -n airwave-dev -it busybox --image=busybox --rm -- sh
wget --spider http://nginx.default

If all went well, you should be able to access the nginx-for-calicoservice and pod from the busyboxinstance in the airwave-dev namespace. You’ll get a remote file exists response. Leave this terminal alone and open a new session.

Dive into use cases and policies to showcase how it works

Official Documentation: https://kubernetes.io/docs/concepts/services-networking/network-policies/

1. The first use case is blocking all ingress traffic for the default namespace.

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress

Here is a network policy that denies all ingress traffic. Notice how namespace is not defined so by default, it will be applied to the context you are in. In my case, it’s the default namespace.

The YAML is straightforward except for podSelector. We will get more into that later, but right now, just know that {} means that the network policy applies to all pods in the namespace it’s being applied to.

In your new terminal session, apply this to your cluster and it will block all ingress traffic to that namespace.

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/deny-default-ingress.yaml

Now that the networking policy has been applied, you should not be able to hit the Nginx container from the busyboxpod in the airwave-dev namespace. Let’s try it again.

Run the busyboxpod if it’s not still open in your terminal and try the wget command again. It should time out because we blocked all ingress traffic going into default.

kubectl run -n airwave-dev -it busybox --image=busybox --rm -- sh
wget --spider http://nginx-for-calico-demo.default

Congrats, you just set up your first networking policy! You have blocked all ingress traffic from going into the default namespace.

The situation we’re in now.

2. For our 2nd use case, let’s block egress traffic for the namespace airwave-dev.

Notice below in the egress policy namespace is defined. This is because we want to block all egress traffic coming out of the airwave-dev namespace, which would include the busyboxpod.

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
  namespace: airwave-dev
spec:
  podSelector: {}
  policyTypes:
  - Egress

Again, podSelector: {}applies to every pod in the airwave-dev namespace. Apply the egress policy:

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/deny-airwave-dev-egress.yaml

In the busyboxinteractive terminal session, try any command that relies on outbound network access. In my example, I’m using nslookup.

nslookup www.google.com

Congrats! You now blocked all egress traffic coming from the airwave-dev namespace.

Our current state

3. Our third example will allow traffic to nginx from our busybox pod in the airwave-dev namespace, but will still block all the other traffic.

This is what the end result will look like

In order to achieve this, we will need to make a few changes.

• Label the namespaces
• Allow DNS lookups using an egress policy to kube-system namespace
• An ingress policy allowing nginxto receive traffic from busybox
• An egress policy allowing busyboxto send traffic to nginx

Label the Namespaces

The reason we need to label the namespaces is that the network policies currently don’t allow targeting namespaces by their name. Since they allow targeting namespaces by labels, let’s create a few.

kubectl label namespace airwave-dev ns=airwave-dev
kubectl label namespace kube-system ns=kube-system
kubectl label namespace default ns=default

Allow DNS using an egress policy

Allowing DNS traffic from the airwave-dev namespace to the kube-system namespace.

The following YAML allows for DNS access from the airwave-dev namespace to kube-system . This is necessary because we are accessing the DNS name of the service (nginx-for-calico-demo.default) and the Kubernetes CoreDNS service lives in the kube-system namespace. We were accessing everything by IP, this step wouldn’t be necessary.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-access
  namespace: airwave-dev 
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          ns: kube-system
    ports:
    - protocol: UDP
      port: 53

We are introduced to namespaceSelectorand ports. These are the ways to target your egress destination. In other words, this policy allows traffic from any pod in the airwave-dev namespace, to any pod within a namespace labeled ns=kube-systemover UDP port 53.

Apply the policy and test nslookupagain.

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-dns.yaml

An ingress policy allowing nginx to receive traffic from busybox

Network policies are unidirectional, meaning if you have to specifically state ingress and egress rules for source and destination traffic.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-from-busybox
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: nginx
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          ns: airwave-dev

In our example, nginx-for-calico-demo has a label app=nginx . The policy above states in the default namespace, pods with the label app=nginx are allowed ingress traffic from all pods in the namespaces with the label ns=airwave-dev .

kubectl get deployment -o wide to check my labels.

As mentioned earlier, we have podSelector. This allows associating network policies with pods and their labels. The usage of labels here is similar to using labels and selectors with services, and with affinity.

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-ingress-from-busybox.yaml

At this point, if you try to access the Nginx server, you will still get a timeout. That’s because we need to create an egress rule for pods in the airwave-dev namespace.

An egress policy allowing busybox to send traffic to nginx

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-egress-from-busybox
  namespace: airwave-dev
spec:
  podSelector: {}
  policyTypes: 
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          ns: default
      podSelector:
        matchLabels:
          app: nginx

This policy states we are allowing all traffic from the airwave-dev namespace to a pod with the label app=nginx, within a namespace labeled ns=default.

kubectl apply -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-egress-to-nginx.yaml

Wrapping up

If you’ve made it this far, there’s one more thing to do. Let’s test the network policies.

Our current setup

In your interactive shell:

• Access the Nginxweb server from the busyboxinstance. According to the diagram above, this should work.

wget http://nginx-for-calico-demo.default --spider

• Access DNS from your busyboxinstance. According to the diagram above, this should work.

nslookup www.google.com

• Access an external website from the busyboxinstance. According to the diagram above, this should NOT work.

wget --spider https://www.google.com

UNINSTALLING

If you’d like to roll back any of the changes, use the following:

kubectl delete -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-dns.yaml
kubectl delete -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-ingress-from-busybox.yaml
kubectl delete -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/allow-egress-to-nginx.yaml
kubectl delete -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/deny-default-ingress.yaml
kubectl delete -f https://raw.githubusercontent.com/airwavetechio/calico-istio-demo/main/policies/deny-airwave-dev-egress.yaml
kubectl delete ns airwave-dev
kubectl delete deployment nginx-for-calico-demo

That will conclude our lesson for today. I hope you were able to follow along and gain more insight into how network policies can be used in Kubernetes.