64 lines
2.3 KiB
Markdown
64 lines
2.3 KiB
Markdown
---
|
|
title: "Day 3 — Kubernetes on Bare Metal, Flux GitOps, and Why I Stopped Using k3s"
|
|
description: "Graduating from k3s to full kubeadm, setting up Flux CD for GitOps, and the first taste of what Longhorn storage actually means."
|
|
pubDate: 2024-11-15
|
|
day: 3
|
|
tags: ["kubernetes", "flux", "gitops", "longhorn", "k8s"]
|
|
---
|
|
|
|
## Why Not k3s Forever?
|
|
|
|
k3s is excellent. I used it for two months and it worked fine. I moved to full Kubernetes (kubeadm) for one reason: I wanted the experience to transfer directly to production environments. At work we don't run k3s. The extra complexity of kubeadm is the point.
|
|
|
|
## The Install
|
|
|
|
```bash
|
|
# kubeadm init on control plane
|
|
sudo kubeadm init \
|
|
--pod-network-cidr=10.244.0.0/16 \
|
|
--control-plane-endpoint="k8s-control.int.h0melab.uk"
|
|
|
|
# CNI — went with Cilium over Flannel for eBPF goodness
|
|
helm install cilium cilium/cilium --namespace kube-system
|
|
```
|
|
|
|
## Flux GitOps
|
|
|
|
This was the decision that changed everything. Instead of `kubectl apply`-ing manifests, every change goes through Git:
|
|
|
|
```
|
|
homelab-k8s/
|
|
├── clusters/homelab/
|
|
│ ├── flux-system/ ← Flux's own manifests
|
|
│ ├── infrastructure/ ← Traefik, Longhorn, cert-manager
|
|
│ └── apps/ ← Actual workloads
|
|
```
|
|
|
|
The golden rule I established here: **Chart.yaml version bumps are required for Flux to pick up Helm chart changes.** Forgot this approximately 15 times before it became instinct.
|
|
|
|
## Longhorn
|
|
|
|
Distributed block storage across three worker nodes. The UI is surprisingly good. The first time I watched a volume replica heal itself after a node reboot, I understood why people write blog posts about storage.
|
|
|
|
```yaml
|
|
# The PVC pattern I use for everything
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: app-data
|
|
annotations:
|
|
helm.sh/resource-policy: keep # ← never delete this on helm uninstall
|
|
spec:
|
|
storageClassName: longhorn
|
|
accessModes: [ReadWriteOnce]
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
```
|
|
|
|
The `helm.sh/resource-policy: keep` annotation saved my data at least twice when I was iterating on Helm releases.
|
|
|
|
## What's Next
|
|
|
|
Day 4 covers the monitoring stack — Netdata agents, VictoriaMetrics as the TSDB, and getting Grafana to look like something I'd actually want to stare at during an incident.
|