From 083478f980fdd3c3096f3d87d798ca197e78c288 Mon Sep 17 00:00:00 2001 From: duynguyen Date: Sun, 14 Jun 2026 16:55:40 +0700 Subject: [PATCH] add README with step-by-step bootstrap instructions --- README.md | 292 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 292 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..a1603c4 --- /dev/null +++ b/README.md @@ -0,0 +1,292 @@ +# K8s Cluster Bootstrap — Platform Services + +Bootstraps platform services onto the k8s cluster using ArgoCD app-of-apps pattern. +Run this after cluster-init finishes provisioning and configuring nodes. + +## Architecture + +``` +Git repo (cluster-bootstrap) + └── ArgoCD watches bootstrap/apps/ → syncs all Applications + +Bootstrap order (sync waves): + Wave 0 → metallb (LoadBalancer IPs) + Wave 1 → metallb-config (IPAddressPool + L2Advertisement) + Wave 2 → envoy-gateway (HTTP gateway controller) + Wave 2 → nfs-provisioner (dynamic PVC provisioner from xpen NAS) + Wave 3 → envoy-gateway-config (GatewayClass + EnvoyProxy + Gateway) + +External access: + MetalLB assigns 192.168.1.30 to Envoy Gateway LoadBalancer service + All HTTP traffic → Envoy Gateway (192.168.1.30:80) → HTTPRoutes → services + DNS: *.fireflylab.local → 192.168.1.30 (configure in your local DNS/router) +``` + +--- + +## Prerequisites + +- k8s cluster running (see cluster-init repo) +- `kubectl` configured on client machine (kubeconfig at `~/.kube/config`) +- Client machine can reach `192.168.1.31` (master01) +- xpen NAS NFS export accessible from all k8s nodes + +--- + +## Phase 1 — Install Helm + +```bash +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +``` + +```bash +helm version +``` + +--- + +## Phase 2 — Fill in NFS details + +Edit `manifests/nfs-provisioner/values.yaml` before pushing — replace the two placeholders: + +```bash +git clone https://gitea.fireflylab.cc/duynguyen/cluster-bootstrap.git +``` + +```bash +cd cluster-bootstrap +``` + +Open `manifests/nfs-provisioner/values.yaml` and set: +- `nfs.server` — xpen NAS IP address +- `nfs.path` — NFS export path (e.g. `/volume1/k8s`) + +Then commit and push: + +```bash +git add manifests/nfs-provisioner/values.yaml +``` + +```bash +git commit -m "set nfs server and path" +``` + +```bash +git push origin main +``` + +--- + +## Phase 3 — Install ArgoCD + +Add Helm repo: + +```bash +helm repo add argo https://argoproj.github.io/argo-helm +``` + +```bash +helm repo update +``` + +Install ArgoCD: + +```bash +helm install argocd argo/argo-cd -n argocd --create-namespace -f manifests/argocd/values.yaml +``` + +Wait for ArgoCD to be ready: + +```bash +kubectl wait --for=condition=available deployment/argocd-server -n argocd --timeout=120s +``` + +Get initial admin password: + +```bash +kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath='{.data.password}' | base64 -d +``` + +Port-forward to access UI (MetalLB + Envoy not running yet): + +```bash +kubectl port-forward svc/argocd-server -n argocd 8080:443 +``` + +Open `https://localhost:8080` — login with `admin` and the password above. + +--- + +## Phase 4 — Create bootstrap app in ArgoCD UI + +In ArgoCD UI, create a new Application: + +| Field | Value | +|-------|-------| +| Application Name | `bootstrap` | +| Project | `default` | +| Sync Policy | Automated (enable Prune + Self Heal) | +| Repository URL | `https://gitea.fireflylab.cc/duynguyen/cluster-bootstrap.git` | +| Revision | `main` | +| Path | `bootstrap/apps` | +| Cluster URL | `https://kubernetes.default.svc` | +| Namespace | `argocd` | + +Click **Create**. ArgoCD syncs all child Applications in wave order automatically. + +Monitor progress: + +```bash +kubectl get applications -n argocd +``` + +```bash +kubectl get pods -n metallb-system +``` + +```bash +kubectl get pods -n envoy-gateway-system +``` + +```bash +kubectl get pods -n nfs-provisioner +``` + +--- + +## Phase 5 — Verify Envoy Gateway has external IP + +```bash +kubectl get svc -n envoy-gateway-system +``` + +`EXTERNAL-IP` should be `192.168.1.30` (assigned by MetalLB). + +If it stays ``, check MetalLB: + +```bash +kubectl get ipaddresspool -n metallb-system +``` + +```bash +kubectl get l2advertisement -n metallb-system +``` + +--- + +## Phase 6 — Apply ArgoCD HTTPRoute + +Once Envoy Gateway has the external IP, expose ArgoCD via hostname: + +```bash +kubectl apply -f manifests/argocd/httproute.yaml +``` + +ArgoCD UI now accessible at `http://argocd.fireflylab.local` — no more port-forward needed. + +--- + +## Phase 7 — Verify StorageClasses + +```bash +kubectl get storageclass +``` + +Expected: + +``` +NAME PROVISIONER RECLAIMPOLICY +nfs-delete cluster.local/nfs-subdir-external-provisioner Delete +nfs-retain cluster.local/nfs-subdir-external-provisioner Retain +``` + +Test PVC provisioning: + +```bash +kubectl apply -f - <<'EOF' +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: nfs-test + namespace: default +spec: + accessModes: [ReadWriteOnce] + storageClassName: nfs-delete + resources: + requests: + storage: 1Gi +EOF +``` + +```bash +kubectl get pvc nfs-test +``` + +Status should be `Bound`. Clean up: + +```bash +kubectl delete pvc nfs-test +``` + +--- + +## Full verification + +```bash +kubectl get applications -n argocd +``` + +```bash +kubectl get pods -n metallb-system +``` + +```bash +kubectl get pods -n envoy-gateway-system +``` + +```bash +kubectl get pods -n nfs-provisioner +``` + +```bash +kubectl get svc -n envoy-gateway-system +``` + +```bash +kubectl get storageclass +``` + +All Applications should be `Synced` / `Healthy`. + +--- + +## File reference + +``` +cluster-bootstrap/ +├── README.md +├── bootstrap/ +│ └── apps/ # ArgoCD Application CRDs +│ ├── metallb.yaml # wave 0 — Helm chart +│ ├── metallb-config.yaml # wave 1 — IPAddressPool + L2Advertisement +│ ├── envoy-gateway.yaml # wave 2 — Helm chart +│ ├── nfs-provisioner.yaml # wave 2 — Helm chart +│ └── envoy-gateway-config.yaml # wave 3 — GatewayClass + EnvoyProxy + Gateway +└── manifests/ + ├── argocd/ + │ ├── values.yaml # ArgoCD Helm values + │ └── httproute.yaml # ArgoCD HTTPRoute (applied after Envoy is up) + ├── metallb/ + │ └── values.yaml + ├── metallb-config/ + │ ├── ipaddresspool.yaml # IP pool: 192.168.1.30/32 + │ └── l2advertisement.yaml + ├── envoy-gateway/ + │ └── values.yaml + ├── envoy-gateway-config/ + │ ├── gatewayclass.yaml + │ ├── envoy-proxy.yaml # DaemonSet, LoadBalancer service + │ └── gateway.yaml # HTTP :80 listener + └── nfs-provisioner/ + └── values.yaml # ⚠ fill in nfs.server + nfs.path before push +```