add README with step-by-step bootstrap instructions
This commit is contained in:
292
README.md
Normal file
292
README.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# K8s Cluster Bootstrap — Platform Services
|
||||
|
||||
Bootstraps platform services onto the k8s cluster using ArgoCD app-of-apps pattern.
|
||||
Run this after cluster-init finishes provisioning and configuring nodes.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Git repo (cluster-bootstrap)
|
||||
└── ArgoCD watches bootstrap/apps/ → syncs all Applications
|
||||
|
||||
Bootstrap order (sync waves):
|
||||
Wave 0 → metallb (LoadBalancer IPs)
|
||||
Wave 1 → metallb-config (IPAddressPool + L2Advertisement)
|
||||
Wave 2 → envoy-gateway (HTTP gateway controller)
|
||||
Wave 2 → nfs-provisioner (dynamic PVC provisioner from xpen NAS)
|
||||
Wave 3 → envoy-gateway-config (GatewayClass + EnvoyProxy + Gateway)
|
||||
|
||||
External access:
|
||||
MetalLB assigns 192.168.1.30 to Envoy Gateway LoadBalancer service
|
||||
All HTTP traffic → Envoy Gateway (192.168.1.30:80) → HTTPRoutes → services
|
||||
DNS: *.fireflylab.local → 192.168.1.30 (configure in your local DNS/router)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- k8s cluster running (see cluster-init repo)
|
||||
- `kubectl` configured on client machine (kubeconfig at `~/.kube/config`)
|
||||
- Client machine can reach `192.168.1.31` (master01)
|
||||
- xpen NAS NFS export accessible from all k8s nodes
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Install Helm
|
||||
|
||||
```bash
|
||||
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
|
||||
```
|
||||
|
||||
```bash
|
||||
helm version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Fill in NFS details
|
||||
|
||||
Edit `manifests/nfs-provisioner/values.yaml` before pushing — replace the two placeholders:
|
||||
|
||||
```bash
|
||||
git clone https://gitea.fireflylab.cc/duynguyen/cluster-bootstrap.git
|
||||
```
|
||||
|
||||
```bash
|
||||
cd cluster-bootstrap
|
||||
```
|
||||
|
||||
Open `manifests/nfs-provisioner/values.yaml` and set:
|
||||
- `nfs.server` — xpen NAS IP address
|
||||
- `nfs.path` — NFS export path (e.g. `/volume1/k8s`)
|
||||
|
||||
Then commit and push:
|
||||
|
||||
```bash
|
||||
git add manifests/nfs-provisioner/values.yaml
|
||||
```
|
||||
|
||||
```bash
|
||||
git commit -m "set nfs server and path"
|
||||
```
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Install ArgoCD
|
||||
|
||||
Add Helm repo:
|
||||
|
||||
```bash
|
||||
helm repo add argo https://argoproj.github.io/argo-helm
|
||||
```
|
||||
|
||||
```bash
|
||||
helm repo update
|
||||
```
|
||||
|
||||
Install ArgoCD:
|
||||
|
||||
```bash
|
||||
helm install argocd argo/argo-cd -n argocd --create-namespace -f manifests/argocd/values.yaml
|
||||
```
|
||||
|
||||
Wait for ArgoCD to be ready:
|
||||
|
||||
```bash
|
||||
kubectl wait --for=condition=available deployment/argocd-server -n argocd --timeout=120s
|
||||
```
|
||||
|
||||
Get initial admin password:
|
||||
|
||||
```bash
|
||||
kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath='{.data.password}' | base64 -d
|
||||
```
|
||||
|
||||
Port-forward to access UI (MetalLB + Envoy not running yet):
|
||||
|
||||
```bash
|
||||
kubectl port-forward svc/argocd-server -n argocd 8080:443
|
||||
```
|
||||
|
||||
Open `https://localhost:8080` — login with `admin` and the password above.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Create bootstrap app in ArgoCD UI
|
||||
|
||||
In ArgoCD UI, create a new Application:
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Application Name | `bootstrap` |
|
||||
| Project | `default` |
|
||||
| Sync Policy | Automated (enable Prune + Self Heal) |
|
||||
| Repository URL | `https://gitea.fireflylab.cc/duynguyen/cluster-bootstrap.git` |
|
||||
| Revision | `main` |
|
||||
| Path | `bootstrap/apps` |
|
||||
| Cluster URL | `https://kubernetes.default.svc` |
|
||||
| Namespace | `argocd` |
|
||||
|
||||
Click **Create**. ArgoCD syncs all child Applications in wave order automatically.
|
||||
|
||||
Monitor progress:
|
||||
|
||||
```bash
|
||||
kubectl get applications -n argocd
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n metallb-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n envoy-gateway-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n nfs-provisioner
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Verify Envoy Gateway has external IP
|
||||
|
||||
```bash
|
||||
kubectl get svc -n envoy-gateway-system
|
||||
```
|
||||
|
||||
`EXTERNAL-IP` should be `192.168.1.30` (assigned by MetalLB).
|
||||
|
||||
If it stays `<pending>`, check MetalLB:
|
||||
|
||||
```bash
|
||||
kubectl get ipaddresspool -n metallb-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get l2advertisement -n metallb-system
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Apply ArgoCD HTTPRoute
|
||||
|
||||
Once Envoy Gateway has the external IP, expose ArgoCD via hostname:
|
||||
|
||||
```bash
|
||||
kubectl apply -f manifests/argocd/httproute.yaml
|
||||
```
|
||||
|
||||
ArgoCD UI now accessible at `http://argocd.fireflylab.local` — no more port-forward needed.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Verify StorageClasses
|
||||
|
||||
```bash
|
||||
kubectl get storageclass
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```
|
||||
NAME PROVISIONER RECLAIMPOLICY
|
||||
nfs-delete cluster.local/nfs-subdir-external-provisioner Delete
|
||||
nfs-retain cluster.local/nfs-subdir-external-provisioner Retain
|
||||
```
|
||||
|
||||
Test PVC provisioning:
|
||||
|
||||
```bash
|
||||
kubectl apply -f - <<'EOF'
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: nfs-test
|
||||
namespace: default
|
||||
spec:
|
||||
accessModes: [ReadWriteOnce]
|
||||
storageClassName: nfs-delete
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
EOF
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pvc nfs-test
|
||||
```
|
||||
|
||||
Status should be `Bound`. Clean up:
|
||||
|
||||
```bash
|
||||
kubectl delete pvc nfs-test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full verification
|
||||
|
||||
```bash
|
||||
kubectl get applications -n argocd
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n metallb-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n envoy-gateway-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get pods -n nfs-provisioner
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get svc -n envoy-gateway-system
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl get storageclass
|
||||
```
|
||||
|
||||
All Applications should be `Synced` / `Healthy`.
|
||||
|
||||
---
|
||||
|
||||
## File reference
|
||||
|
||||
```
|
||||
cluster-bootstrap/
|
||||
├── README.md
|
||||
├── bootstrap/
|
||||
│ └── apps/ # ArgoCD Application CRDs
|
||||
│ ├── metallb.yaml # wave 0 — Helm chart
|
||||
│ ├── metallb-config.yaml # wave 1 — IPAddressPool + L2Advertisement
|
||||
│ ├── envoy-gateway.yaml # wave 2 — Helm chart
|
||||
│ ├── nfs-provisioner.yaml # wave 2 — Helm chart
|
||||
│ └── envoy-gateway-config.yaml # wave 3 — GatewayClass + EnvoyProxy + Gateway
|
||||
└── manifests/
|
||||
├── argocd/
|
||||
│ ├── values.yaml # ArgoCD Helm values
|
||||
│ └── httproute.yaml # ArgoCD HTTPRoute (applied after Envoy is up)
|
||||
├── metallb/
|
||||
│ └── values.yaml
|
||||
├── metallb-config/
|
||||
│ ├── ipaddresspool.yaml # IP pool: 192.168.1.30/32
|
||||
│ └── l2advertisement.yaml
|
||||
├── envoy-gateway/
|
||||
│ └── values.yaml
|
||||
├── envoy-gateway-config/
|
||||
│ ├── gatewayclass.yaml
|
||||
│ ├── envoy-proxy.yaml # DaemonSet, LoadBalancer service
|
||||
│ └── gateway.yaml # HTTP :80 listener
|
||||
└── nfs-provisioner/
|
||||
└── values.yaml # ⚠ fill in nfs.server + nfs.path before push
|
||||
```
|
||||
Reference in New Issue
Block a user