---
title: "Day 3 — Kubernetes on Bare Metal, Flux GitOps, and Why I Stopped Using k3s"
description: "Graduating from k3s to full kubeadm, setting up Flux CD for GitOps, and the first taste of what Longhorn storage actually means."
pubDate: 2024-11-15
day: 3
tags: ["kubernetes", "flux", "gitops", "longhorn", "k8s"]
---

## Why Not k3s Forever?

k3s is excellent. I used it for two months and it worked fine. I moved to full Kubernetes (kubeadm) for one reason: I wanted the experience to transfer directly to production environments. At work we don't run k3s. The extra complexity of kubeadm is the point.

## The Install

```bash
# kubeadm init on control plane
sudo kubeadm init \
  --pod-network-cidr=10.244.0.0/16 \
  --control-plane-endpoint="k8s-control.int.h0melab.uk"

# CNI — went with Cilium over Flannel for eBPF goodness
helm install cilium cilium/cilium --namespace kube-system
```

## Flux GitOps

This was the decision that changed everything. Instead of `kubectl apply`-ing manifests, every change goes through Git:

```
homelab-k8s/
├── clusters/homelab/
│   ├── flux-system/      ← Flux's own manifests
│   ├── infrastructure/   ← Traefik, Longhorn, cert-manager
│   └── apps/             ← Actual workloads
```

The golden rule I established here: **Chart.yaml version bumps are required for Flux to pick up Helm chart changes.** Forgot this approximately 15 times before it became instinct.

## Longhorn

Distributed block storage across three worker nodes. The UI is surprisingly good. The first time I watched a volume replica heal itself after a node reboot, I understood why people write blog posts about storage.

```yaml
# The PVC pattern I use for everything
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-data
  annotations:
    helm.sh/resource-policy: keep   # ← never delete this on helm uninstall
spec:
  storageClassName: longhorn
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 10Gi
```

The `helm.sh/resource-policy: keep` annotation saved my data at least twice when I was iterating on Helm releases.

## What's Next

Day 4 covers the monitoring stack — Netdata agents, VictoriaMetrics as the TSDB, and getting Grafana to look like something I'd actually want to stare at during an incident.