2.3 KiB
title, description, pubDate, day, tags
| title | description | pubDate | day | tags | |||||
|---|---|---|---|---|---|---|---|---|---|
| Day 3 — Kubernetes on Bare Metal, Flux GitOps, and Why I Stopped Using k3s | Graduating from k3s to full kubeadm, setting up Flux CD for GitOps, and the first taste of what Longhorn storage actually means. | 2024-11-15 | 3 |
|
Why Not k3s Forever?
k3s is excellent. I used it for two months and it worked fine. I moved to full Kubernetes (kubeadm) for one reason: I wanted the experience to transfer directly to production environments. At work we don't run k3s. The extra complexity of kubeadm is the point.
The Install
# kubeadm init on control plane
sudo kubeadm init \
--pod-network-cidr=10.244.0.0/16 \
--control-plane-endpoint="k8s-control.int.h0melab.uk"
# CNI — went with Cilium over Flannel for eBPF goodness
helm install cilium cilium/cilium --namespace kube-system
Flux GitOps
This was the decision that changed everything. Instead of kubectl apply-ing manifests, every change goes through Git:
homelab-k8s/
├── clusters/homelab/
│ ├── flux-system/ ← Flux's own manifests
│ ├── infrastructure/ ← Traefik, Longhorn, cert-manager
│ └── apps/ ← Actual workloads
The golden rule I established here: Chart.yaml version bumps are required for Flux to pick up Helm chart changes. Forgot this approximately 15 times before it became instinct.
Longhorn
Distributed block storage across three worker nodes. The UI is surprisingly good. The first time I watched a volume replica heal itself after a node reboot, I understood why people write blog posts about storage.
# The PVC pattern I use for everything
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: app-data
annotations:
helm.sh/resource-policy: keep # ← never delete this on helm uninstall
spec:
storageClassName: longhorn
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 10Gi
The helm.sh/resource-policy: keep annotation saved my data at least twice when I was iterating on Helm releases.
What's Next
Day 4 covers the monitoring stack — Netdata agents, VictoriaMetrics as the TSDB, and getting Grafana to look like something I'd actually want to stare at during an incident.