cluster-bootstrap/README.md

# K8s Cluster Bootstrap — Platform Services

Bootstraps platform services onto the k8s cluster using ArgoCD app-of-apps pattern.
Run this after cluster-init finishes provisioning and configuring nodes.

## Architecture

```
Git repo (cluster-bootstrap)
  └── ArgoCD watches bootstrap/apps/ → syncs all Applications

Bootstrap order (sync waves):
  Wave 0 → metallb          (LoadBalancer IPs)
  Wave 1 → metallb-config   (IPAddressPool + L2Advertisement)
  Wave 2 → envoy-gateway    (HTTP gateway controller)
  Wave 2 → nfs-provisioner  (dynamic PVC provisioner from xpen NAS)
  Wave 3 → envoy-gateway-config (GatewayClass + EnvoyProxy + Gateway)

External access:
  MetalLB assigns 192.168.1.30 to Envoy Gateway LoadBalancer service
  All HTTP traffic → Envoy Gateway (192.168.1.30:80) → HTTPRoutes → services
  DNS: *.fireflylab.local → 192.168.1.30  (configure in your local DNS/router)
```

---

## Prerequisites

- k8s cluster running (see cluster-init repo)
- `kubectl` configured on client machine (kubeconfig at `~/.kube/config`)
- Client machine can reach `192.168.1.31` (master01)
- xpen NAS NFS export accessible from all k8s nodes

---

## Phase 1 — Install Helm

```bash
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
```

```bash
helm version
```

---

## Phase 2 — Set credentials

Install tools (one-time):

```bash
sudo dnf install -y httpd-tools gettext
```

Copy and fill in `.env`:

```bash
cp .env.example .env
```

Generate bcrypt hash and set it in `.env`:

```bash
htpasswd -nbBC 10 "" YOUR_PASSWORD | tr -d ':\n' | sed 's/$2y/$2a/'
```

`.env` is gitignored — never commit it.

---

## Phase 3 — Install ArgoCD

Add Helm repo:

```bash
helm repo add argo https://argoproj.github.io/argo-helm
```

```bash
helm repo update
```

Install ArgoCD (`envsubst` fills `$ARGOCD_ADMIN_PASSWORD_HASH` from `.env` before helm reads the values):

```bash
source .env
```

```bash
envsubst < manifests/argocd/values.yaml | helm install argocd argo/argo-cd -n argocd --create-namespace -f -
```

Wait for ArgoCD to be ready:

```bash
kubectl wait --for=condition=available deployment/argocd-server -n argocd --timeout=120s
```

Port-forward to access UI (MetalLB + Envoy not running yet):

```bash
kubectl port-forward svc/argocd-server -n argocd 8080:443
```

Open `https://localhost:8080` — login with `admin` and the password you chose.

### 3.1 Bake the hash into values.yaml for self-management

ArgoCD won't run `envsubst` when it self-manages — replace the placeholder with the real hash so future syncs work:

```bash
source .env
sed -i "s|\$ARGOCD_ADMIN_PASSWORD_HASH|$ARGOCD_ADMIN_PASSWORD_HASH|" manifests/argocd/values.yaml
```

```bash
git add manifests/argocd/values.yaml && git commit -m "set argocd admin password hash"
```

```bash
git push origin main
```

---

## Phase 4 — Create bootstrap app in ArgoCD UI

In ArgoCD UI, create a new Application:

| Field | Value |
|-------|-------|
| Application Name | `bootstrap` |
| Project | `default` |
| Sync Policy | Automated (enable Prune + Self Heal) |
| Repository URL | `https://gitea.fireflylab.cc/duynguyen/cluster-bootstrap.git` |
| Revision | `main` |
| Path | `bootstrap/apps` |
| Cluster URL | `https://kubernetes.default.svc` |
| Namespace | `argocd` |

Click **Create**. ArgoCD syncs all child Applications in wave order automatically.

Monitor progress:

```bash
kubectl get applications -n argocd
```

```bash
kubectl get pods -n metallb-system
```

```bash
kubectl get pods -n envoy-gateway-system
```

```bash
kubectl get pods -n nfs-provisioner
```

---

## Phase 5 — Verify Envoy Gateway has external IP

```bash
kubectl get svc -n envoy-gateway-system
```

`EXTERNAL-IP` should be `192.168.1.30` (assigned by MetalLB).

If it stays `<pending>`, check MetalLB:

```bash
kubectl get ipaddresspool -n metallb-system
```

```bash
kubectl get l2advertisement -n metallb-system
```

---

## Phase 6 — Apply ArgoCD HTTPRoute

Once Envoy Gateway has the external IP, expose ArgoCD via hostname:

```bash
kubectl apply -f manifests/argocd/httproute.yaml
```

ArgoCD UI now accessible at `http://argocd.fireflylab.local` — no more port-forward needed.

---

## Phase 7 — Verify StorageClasses

```bash
kubectl get storageclass
```

Expected:

```
NAME          PROVISIONER                                     RECLAIMPOLICY
nfs-delete    cluster.local/nfs-subdir-external-provisioner  Delete
nfs-retain    cluster.local/nfs-subdir-external-provisioner  Retain
```

Test PVC provisioning:

```bash
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-test
  namespace: default
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: nfs-delete
  resources:
    requests:
      storage: 1Gi
EOF
```

```bash
kubectl get pvc nfs-test
```

Status should be `Bound`. Clean up:

```bash
kubectl delete pvc nfs-test
```

---

## Full verification

```bash
kubectl get applications -n argocd
```

```bash
kubectl get pods -n metallb-system
```

```bash
kubectl get pods -n envoy-gateway-system
```

```bash
kubectl get pods -n nfs-provisioner
```

```bash
kubectl get svc -n envoy-gateway-system
```

```bash
kubectl get storageclass
```

All Applications should be `Synced` / `Healthy`.

---

## File reference

```
cluster-bootstrap/
├── README.md
├── bootstrap/
│   └── apps/                         # ArgoCD Application CRDs
│       ├── metallb.yaml              # wave 0 — Helm chart
│       ├── metallb-config.yaml       # wave 1 — IPAddressPool + L2Advertisement
│       ├── envoy-gateway.yaml        # wave 2 — Helm chart
│       ├── nfs-provisioner.yaml      # wave 2 — Helm chart
│       └── envoy-gateway-config.yaml # wave 3 — GatewayClass + EnvoyProxy + Gateway
└── manifests/
    ├── argocd/
    │   ├── values.yaml               # ArgoCD Helm values
    │   └── httproute.yaml            # ArgoCD HTTPRoute (applied after Envoy is up)
    ├── metallb/
    │   └── values.yaml
    ├── metallb-config/
    │   ├── ipaddresspool.yaml        # IP pool: 192.168.1.30/32
    │   └── l2advertisement.yaml
    ├── envoy-gateway/
    │   └── values.yaml
    ├── envoy-gateway-config/
    │   ├── gatewayclass.yaml
    │   ├── envoy-proxy.yaml          # DaemonSet, LoadBalancer service
    │   └── gateway.yaml             # HTTP :80 listener
    └── nfs-provisioner/
        └── values.yaml              # ⚠ fill in nfs.server + nfs.path before push
```