GitOps with Flux and Kustomize
On Windows 10 with minikube & different namespaces
So far, we have covered GitOps in theory. Now, we are going to put GitOps into practice using Weaveworks Flux and Kustomize.
We are going to set up our hello-world example so that every time you push a change to a particular branch in your git repository, those changes will sync to Kubernetes, or in our case minikube, and ensure your application is in the state that you want it to be in. We are going to use Kustomize to highlight how we could deploy this service to different namespaces with different configuration values.
A quick recap of what we will be doing.
- You will need to select a GitHub repository of an application or service you want to sync. You will need access to save a deploy key in GitHub for that repo. My repo as an example: https://github.com/airwavetechio/hello-world
- We are going to install & deploy Flux into a minikube cluster. This will point to your GitHub repository. You should have minikube up and running before starting this exercise.
- Solidify the connection from Flux to the repository by adding a deploy key that is created after you install flux in your cluster.
- Sync your repo with your cluster and watch Kustomize take care of applying those changes.
2. Install & Deploy Flux
We are going to skip step 1 and jump right into step 2.
Flux is an open-source tool created by Weaveworks. It gets deployed to your cluster and is responsible for watching/syncing changes from GitHub to your cluster.
When we install flux, we add command line parameters to properly configure it.
First, let’s install fluxctl
on our local machine. Much like kubectl
, this is your command-line tool to administer flux.
choco install fluxctl
for other platforms, follow https://docs.fluxcd.io/en/1.18.0/references/fluxctl.html#
Let’s create a namespace for flux.
kubectl create ns flux
Let’s install flux on your cluster and configure it.
fluxctl install --manifest-generation=true --git-user=<a git user> --git-email=<email used for git> --git-url=git@github.com:org/<repo> --git-branch <branch> --git-path=<path to your files, additional path> --namespace=<namespace> | kubectl apply -f -
--manifest-generation=true
is going to allow us to leverage a bit of templating so we can reuse a base set of YAML files for Kubernetes and patch the changes with Kustomize. This will help us when deploying specific app configurations for each namespace.
--git_user=<a git user> --git-email=<email used for git> --git-url=git@github.com:org/<repo> --git-branch <branch>
is the configuration for your git repo that you want to sync.
--git-path=<path to your files, additional path>
is telling flux where to look for particular files after the repository is cloned (inside of the cluster). I will get into more detail about the path after this section.
--namespace=flux
tells fluxctl
where to install flux on Kubernetes.
| kubectl apply -f -
this pipes the output of the fluxctl
command and applies that output to a kubectl apply
command. (Don’t forget | ).
An explanation of git-path
--git-path=flux/releases/namespaces,flux/releases/airwave-stage
When flux is working, it will clone the specified git repo inside the cluster, and then try to run Kustomize to generate Kubernetes manifest files based on what is located inside the--git-path
. Kustomize will then apply those manifests after they have generated.
In our case here, we have specified 2 locations for Kustomize to look, separated by a comma.
flux\releases\namespaces
and
flux\releases\airwave-stage
Let’s look at our directory structure. We have a base
set of files, along with a directory for each namespace, along with a namespaces
directory. airwave-deploy
will not be used in this example.
In our case, flux\releases
is our root dir. At that level of the tree, we have.flux.yaml
. This is required by flux if you want to use a generator, or else flux will apply YAML files just as kubectl
would. The flux\releases
parent directory is something I just made up to house the files and can be whatever you want.
.flux.yaml
tells flux how it will generate files. You want to drop this at the root of where your synced files will live. That’s because, by default, flux will look in the directory that is specified in--git-path
and the directory above it.
flux\releases\namespaces
I have added namespaces as an example of how to setup namespace the #GitOps way.
By default, Kustomize looks for the file name kustomization.yaml
. In our --git-path
we specified flux\releases\namespaces
, so it will use the kustomization.yaml
file in that directory.
namespaces\kustomization.yaml
tells Kustomize which files to apply.
namespace\airwave-stage.yaml
is a YAML file you could also directly apply to Kubernetes to create a namespace. Remember, flux and Kustomize are syncing the files between GitHub and the cluster for you and applying them automatically, so any YAML files you are using now to configure your services can probably be reused.
flux\releases\base\
base
contains all the base files. In this case, our deployment, service, and configmap YAML files. The files mentioned below can be found in our hello-word repo.
base\kustomization.yaml
lets Kustomize know which files to apply.
base\kustomization.yaml
In our base\deployment.yaml
file, we are NOT specifying a namespace.
base\service.yaml
also does NOT specify a namespace, but specifies certain ports to listen on.
base\configmap.yaml
specifies a default AIRWAVE_HW_PORT
of 4997
that the deployment will look for.
But what does base
have to do with our --git-path
? Let’s find out.
flux\releases\airwave-stage
In this use case, we want to deploy hello-world
to the airwave-stage
namespace.
airwave-stage
’s deployment needs to use port 4998
, not 4997
as configured in the base
. Our service then needs to serve on port 5000
and not 4997
as originally configured.
airwave-stage\kustomization.yaml
again tells Kustomize to use the relative path to the base
directory. It directs flux to install this in the airwave-stage
namespace, and then patch the whatever matches kind:
name:
. It then uses the path:
to specify where the patch file is.
airwave-stage\configmap.yaml
is a patch file. In this case, it looks almost identical to base\configmap.yaml
except for the port. Because it’s a direct match (and short) with the original file, it will just overwrite it.
airwave-stage\service.yaml
is also a patch file, but it’s a little different. Because we need to replace something inline, it wouldn’t make sense to just copy, paste, and make our changes from the original to this file. Here we add $patch: delete
to the fields we want to delete, and then follow it with the new values.
I’m currently not too sure about the state of $patch
because Kustomize v2.03 and v3 are floating around and that makes things a bit confusing. If you find more information on this, let me know but for now, this works.
You should be able to test all of this locally by using kustomize build <directory>
but with these version mismatches, but at the time for writing this article, it doesn’t currently work with v2.0.3 on Windows 10 Pro.
Hopefully, this gives you a better idea of how --git-path
works. Now back to the action.
3. Solidify the connection from Flux to the repository by adding a deploy key
fluxctl identity --k8s-fwd-ns flux
will output an ssh public key that is generated by the Kubernetes cluster. The cluster itself has the private key.
Copy that ssh-key and paste it into your GitHub Repo > Settings > Deploy keys
4. Sync your repo with your cluster
fluxctl --k8s-fwd-ns flux sync
to kick off the process. You can also wait for 5 minutes.
If you did everything right, you will see your changes in the proper area. Let’s look at our diagram again to refresh our memory.
The deployments works.
The configmap is showing port 4998
instead of 4997
Our service is now listening on port 5000
instead of 4997
Conclusion
In this exercise, we have taken our hello-world
service, configured flux and Kustomize in our minikube cluster to sync with our GitHub repo, and have watched an automated deployment of that service to a specific namespace with a specific configuration. Welcome to the future. “This is heavy.”
If you learned something new in this article, don’t forget to clap and follow!
Until next time.