# K8s Cluster Platform — Services

Bootstraps platform/application services (vault, kube-prometheus-stack, harbor,
jenkins, sonarqube, headlamp) onto the cluster using ArgoCD app-of-apps pattern.

Run this after `cluster-bootstrap` finishes (ArgoCD, MetalLB, Envoy Gateway, NFS
storage must already be up).

## Architecture

```
Git repo (cluster-platform)
  └── ArgoCD watches platform/apps/ → syncs all Applications

platform-app.yaml       ← root Application, applied once by hand (kubectl apply)
platform/apps/          ← ArgoCD watches this path; one Application CRD per service
manifests/               ← Helm values + raw manifests referenced by those Applications
```

Sync waves (no hard interdependencies between services yet — grouped for
readability / staggered rollout):

| Wave | Service | Purpose |
|------|---------|---------|
| 0 | vault | Secrets engine (standalone, manual init/unseal) |
| 0 | headlamp | K8s dashboard |
| 1 | headlamp-config | ClusterRoleBinding for login token |
| 1 | external-secrets | Vault → K8s Secret operator |
| 1 | kube-prometheus-stack | Prometheus + Grafana + Alertmanager |
| 1 | harbor | Image registry |
| 2 | external-secrets-config | ClusterSecretStore wired to Vault (k8s auth) |
| 2 | kube-prometheus-stack-config | Grafana HTTPRoute |
| 2 | harbor-config | Harbor HTTPRoute |
| 2 | jenkins | CI |
| 2 | sonarqube | Code quality (embedded H2, no external Postgres) |
| 3 | jenkins-config | Jenkins HTTPRoute |
| 3 | sonarqube-config | SonarQube HTTPRoute |

⚠️ Chart `targetRevision` pins in `platform/apps/*.yaml` are best-effort and
marked `TODO: verify latest` — this session had no live access to the Helm
repos to confirm current versions. Run `helm repo add <name> <url> && helm
search repo <name>/<chart> --versions` before or after first sync and bump
if a pin doesn't resolve.

## Bootstrap Sequence

**Apply the root platform Application**

```bash
kubectl apply -f platform-app.yaml
```

Everything else is reached by ArgoCD syncing `platform/apps/` from here.

### Post-sync manual steps

**Vault — initialize + unseal** (standalone mode, not auto-unseal):

```bash
kubectl exec -n vault vault-0 -- vault operator init -key-shares=1 -key-threshold=1
# save the unseal key + root token shown, then:
kubectl exec -n vault vault-0 -- vault operator unseal <unseal-key>
```

**Wire Vault up for external-secrets** (kv-v2 mount + kubernetes auth method —
this is Vault-internal config, not a k8s resource, so it can't go through
ArgoCD; do it once after unseal):

```bash
kubectl exec -it -n vault vault-0 -- sh
export VAULT_TOKEN=<root-token-from-init>

vault secrets enable -path=kv kv-v2
vault auth enable kubernetes
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc"

vault policy write external-secrets - <<EOF
path "kv/data/*" {
  capabilities = ["read"]
}
EOF

vault write auth/kubernetes/role/external-secrets \
  bound_service_account_names=external-secrets-vault-auth \
  bound_service_account_namespaces=external-secrets \
  policies=external-secrets \
  ttl=1h
```

Once this is done, `external-secrets-config`'s `ClusterSecretStore` (`vault-backend`)
should show `Valid` — check with `kubectl get clustersecretstore vault-backend -o yaml`.

Per-service `ExternalSecret` resources (harbor-credentials, gitea-credentials,
sonarqube-token, Jenkins creds) aren't created yet — that's a follow-up once
you've actually put those secrets into Vault under `kv/`.

**Grafana admin password** (chart auto-generates, never in Git):

```bash
kubectl get secret -n monitoring kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d; echo
```

**Jenkins admin password** (chart auto-generates, never in Git):

```bash
kubectl exec -n jenkins deploy/jenkins -c jenkins -- cat /run/secrets/additional/chart-admin-password; echo
```

**Harbor admin password** — chart ships a default (`Harbor12345`). Log in to
`http://harbor.fireflylab.local` and rotate it immediately; Harbor's password
is set inside its own database on first boot, so it cannot be swapped via a
`kubectl patch` the way ArgoCD's can.

**Headlamp login token** (ServiceAccount created by `headlamp-config`):

```bash
kubectl create token headlamp-admin -n headlamp
```
Paste the token into the Headlamp UI login screen.

### Apply HTTPRoutes note

Each `*-config` Application creates its own HTTPRoute (unlike
`cluster-bootstrap`'s ArgoCD route, which had to be applied by hand to avoid
a chicken-and-egg problem before Envoy existed) — Envoy Gateway is already up
by the time this repo syncs, so these are fully GitOps/auto-synced.

Backend service names in each `httproute.yaml` are best-effort based on each
chart's naming convention and marked with a `verify with: kubectl get svc`
comment — confirm and adjust if a route doesn't resolve.

## Domains (add to local DNS, all → 192.168.1.30)

| Service | Hostname |
|---------|----------|
| Grafana | grafana.fireflylab.local |
| Harbor | harbor.fireflylab.local |
| Jenkins | jenkins.fireflylab.local |
| SonarQube | sonarqube.fireflylab.local |

Vault and Headlamp have no HTTPRoute — Vault stays internal-only
(`vault.vault.svc.cluster.local:8200`); Headlamp is accessed via
`kubectl port-forward` until/unless you add a route for it.