feat: add Helm chart for standalone AI Gateway#27256
Conversation
4e23a5a to
d65696c
Compare
d65696c to
531f6b8
Compare
|
/coder-agents-review |
|
Chat: Review posted | View chat Review historydeep-review v0.9.0 | Round 2 | Last posted: Round 2, 28 findings (1 P2, 12 P3, 1 P4, 8 Nit, 6 Note), COMMENT. Review Finding inventoryFinding inventory - PR #27256Law analysisR1: Effective +2469 -7 (40 files), head 531f6b8. R2: Effective +3054 -7 (51 files, 1001 production / 2053 test), head 2f1347a. Verdict: Don't split (advisory), unchanged both rounds. Growth is schema + docs + golden fixtures within the single concern. Findings
Contested and acknowledgedCRF-9 (P3, templates/ingress.yaml:32) - listenerTLS + ingress plaintext-to-TLS backend
CRF-9 (P3, templates/ingress.yaml:32) - listenerTLS + ingress plaintext-to-TLS backend
CRF-18 (Note, templates/_aigateway.tpl:91) - metrics listener always-on
CRF-22 (Note, templates/_aigateway.tpl:130) - readiness probe coupled to coderd DRPC
CRF-1 (Nit, tests/chart_test.go:75) -
|
| Reviewer | Focus |
|---|---|
| Bisky | tests |
| Chopper | ops/errors |
| Churn-guard | change verification |
| Ging | language modernization |
| Gon | naming |
| Hisoka | edge cases |
| Killua | perf |
| Kite | change integrity |
| Knov | contracts |
| Knuckle | SQL |
| Komugi | flake/determinism |
| Kurapika | security |
| Law | decomposition |
| Leorio | docs |
| Luffy | product |
| Mafu-san | process |
| Mafuuu | contracts |
| Melody | dispatch/pairing |
| Meruem | structural |
| Nami | frontend |
| Netero | mechanical checks |
| Pariston | premise testing |
| Pen-botter | product gaps |
| Razor | verification |
| Robin | duplication |
| Ryosuke | Go arch |
| Takumi | concurrency |
| Zoro | shape |
🤖 Managed by Coder Agents.
There was a problem hiding this comment.
This is a well-built chart. It reuses the libcoder library the way the coder and provisioner charts do, keeps the imperative surface to a few template helpers, ships render-time validation with precise field-level error messages, and backs it with a deterministic golden-file suite (63% test density) that the panel ran green under -race and a GOMAXPROCS sweep. Bisky's take: "this test suite is lovely... These are real gems." No scope drift: every one of the 40 files belongs to the chart or its mechanical build/release wiring, which mirrors the existing chart entries line-for-line.
Severity count: 1 P2, 10 P3, 4 Nit, 6 Note. No P0/P1, so this is a COMMENT, not a block. The one finding to fix before shipping is the liveness/startup race (CRF-4): three reviewers independently traced that the process does not open its /healthz listener until it has connected to coderd and loaded providers, so on a fresh co-install or during a coderd outage the gateway crashloops. The rest cluster into contract-narrowing (CRF-5, CRF-6, CRF-7), operator-signal correctness (CRF-8, CRF-9, CRF-23), and convention drift (CRF-12, CRF-13).
Process notes, not inline findings:
- The commit subject / PR title misspells "helm" as "heml" (CRF-16). It squash-merges into
mainpermanently. Suggestfeat(helm/ai-gateway): add Helm chart for standalone AI Gateway. - The PR description claims the suite covers "all validation error cases," but 6 of the 13
failbranches have no test (this is CRF-2). Mafu-san flagged this as fake completeness: a reviewer trusting the description approves ~46% of the validation logic as tested when it is not. Either add the cases or scope the wording. - Netero's round-1
tc := tcnit was dropped: Komugi empirically proved the copy is load-bearing for the parallel multi-namespace subtests (deleting it failsdefault_valuesdeterministically). Do not remove it.
No findings against the build/release plumbing: the charts list drives the build/ai-gateway_helm_$(VERSION).tgz target before signing, and the attestation ./build/*_helm_*.tgz glob already covers the artifact.
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
There was a problem hiding this comment.
Strong round-2 turnaround. Every one of the 19 round-1 findings the author marked fixed was independently verified against the tree, and the important ones were fixed at the root rather than patched at the symptom: the chart-owned env guard now derives its protected list from the rendered ownedEnv (no more hand-maintained duplicate), automountServiceAccountToken: false moved to the pod spec so it holds on the disableCreate path, extraTemplates delegates to libcoder's coder.renderTemplate again, and the HTTPRoute CRD guard is now actually testable via per-case apiVersions. Bisky: "this test suite is a genuine gem, and I do not say that often" - all 13 validation branches now have a verbatim-message test. TestRenderChart passes.
Contested CRF-9 (listenerTLS behind ingress/httproute): the panel agrees unanimously with the author that a hard validate reject would be wrong, because backend-TLS to the HTTPS listener is legitimate for controllers that re-originate TLS and the chart cannot know the controller's annotation dialect. Kurapika confirmed it fails closed, so there is no security exposure. Five reviewers accept the new README section as a sufficient ceiling. Two (Pariston, Leorio) re-raised a cheap residual: the docs teach the fix but never name the symptom (a 502 / TLS handshake error), and there is no install-time signal even though NOTES.txt already computes the exact condition. Kept open at P3 as a non-blocking doc/UX ask, not a request for validation. See the reply on that thread.
Severity count this round: 2 P3, 1 P4, 4 Nit (all new; no regressions of prior fixes). Two are follow-on consequences of round-1 fixes rather than the original issues re-opening (CRF-27 is the sibling of CRF-7; CRF-28 is guidance that can undo CRF-4).
Acknowledged as intentional by the author and accepted: CRF-18 (metrics listener always-on, matches sibling charts) and CRF-22 (readiness coupled to the coderd DRPC connection).
Process notes:
- The CI
titlecheck is red (✗ title). The current PR title parses clean against the conventional-commit validator, so this may be a stale run after the "heml" -> "Helm" rename, but reviewers could not read the failing run (no GitHub credentials in the worktree). Please confirm it is green before merge rather than assume the rename cleared it; a failing check is a bug until proven stale. - The branch commits still read
feat: add heml chart for standalone AI Gatewayandagentic review 1. The PR title is corrected, so a squash merge lands a clean subject and this is cosmetic, but if the branch is rebase- or merge-committed the "heml" typo reaches main history. Squash, or reword both subjects.
🤖 This review was automatically generated with Coder Agents.
| {{- range (include "coder-ai-gateway.ownedEnv" . | fromYamlArray) }} | ||
| {{- $owned = append $owned .name }} | ||
| {{- end }} | ||
| {{- range .Values.coder.env }} |
There was a problem hiding this comment.
P3 [CRF-27] The env override guard scans only coder.env, so the collision CRF-7 fixed still crashes the container through coder.envFrom. (Hisoka)
The CRF-7 fix loops over .Values.coder.env and rejects any owned name, including CODER_AI_GATEWAY_KEY. But the container also gets .Values.coder.envFrom (_aigateway.tpl:78-81, verified), and the guard never looks there. envFrom keys are opaque at render time, so the loop cannot inspect them, which means this is not fixable by extending the loop.
Two consequences, both verified from the code + CLI:
- The chart always emits
CODER_AI_GATEWAY_KEY_FILE, andresolveAIGatewayKey(enterprise/cli/aigatewaystart.go:366) aborts when bothCODER_AI_GATEWAY_KEYandCODER_AI_GATEWAY_KEY_FILEare set. An operator who shipsCODER_AI_GATEWAY_KEYvia a ConfigMap/Secret referenced bycoder.envFrombypasses the guard and lands in CrashLoopBackOff on the same confusing "mutually exclusive" error. - For the other owned names, Kubernetes gives
envprecedence overenvFrom, so anenvFromoverride is silently ignored, which contradicts thevalues.yaml:4promise "Chart-owned variables cannot be overridden."
The guarantee is only true for coder.env. Fix: scope the values.yaml comment and README to coder.env explicitly and warn that envFrom is not guarded (and that CODER_AI_GATEWAY_KEY there is a startup crash), or drop the absolute wording. Related: the derived owned-set is also dynamic (Mafuuu, Kurapika) so the TLS file-path vars are unguarded when their secret is unset; both reviewers judged that self-inflicted (the referenced file just does not exist), not an attack surface.
🤖
| memory: 1Gi | ||
|
|
||
| # coder.startupProbe -- Startup probe configuration for the AI Gateway. | ||
| # Keep this disabled until /healthz is independent of initial provider load. |
There was a problem hiding this comment.
P3 [CRF-28] The startupProbe comment tells operators to keep it disabled "until /healthz is independent of initial provider load," a condition the architecture never satisfies, and following it while enabling liveness reintroduces the CRF-4 cold-start crashloop. (Mafuuu P3, Pariston Note, Meruem Note)
Both the startupProbe (line 119) and livenessProbe (line 125) blocks carry the identical comment. ai-gateway start never opens its /healthz listener until loadProviders has connected to coderd (aigatewaystart.go: loadProviders blocks, net.Listen runs afterward), so /healthz is coupled to provider load by design and the condition is never met. For liveness the disable is sound and matches the sibling coder chart. For the startup probe the reasoning is inverted: a startupProbe with a generous failureThreshold is precisely the mechanism that makes liveness safe under that ordering. The chart's own custom.yaml fixture proves it (startupProbe enabled, failureThreshold: 12, liveness off).
Consequence: an operator who wants liveness in production reads this comment, keeps startupProbe off, enables livenessProbe, and gets back the ~30s CrashLoopBackOff that CRF-4 removed from the defaults. Meruem adds a second angle: with liveness off there is no restart-on-hang, so a deadlocked (not crashed) replica lingers forever under replicaCount > 1, silently shrinking capacity. Reword the startup-probe comment: enable startupProbe with a generous failureThreshold first, then livenessProbe is safe; /healthz depends on provider load by design, so the startupProbe is required, not deferred. Note that liveness-off means no automatic recovery from a post-startup hang.
🤖
| {{- end }} | ||
| {{- end }} | ||
| {{- end }} | ||
| {{- if .Values.coder.livenessProbe.enabled }} |
There was a problem hiding this comment.
P4 [CRF-29] The livenessProbe render branch is the only one of the three probe blocks with zero golden coverage. (Bisky)
readinessProbe renders in all 9 goldens and startupProbe in 2 (custom, custom_coder), but livenessProbe renders in none: the sole case that sets it, custom.yaml, sets enabled: false, and every other case leaves the default off, so lines 112-124 never execute in the suite. The block is a copy of the startup/readiness blocks with a different values key and path; it reads correctly today, but copy-paste blocks are exactly where a wrong key or a /readyz typo hides, and the startup golden would not catch it. This is a user-enableable surface CRF-14 added. Add coder.livenessProbe.enabled: true to one values case and snapshot it so the block is pinned like its two siblings. P4 because the code is verified correct by reading.
🤖
| keyKey: tls.key | ||
| # The certificate must cover every hostname clients use. | ||
| listenerTLS: | ||
| secretName: "" |
There was a problem hiding this comment.
Nit [CRF-30] Secret references in the aigateway block are inconsistent: three use .name, one uses .secretName. (Gon)
keySecret.name, coderTLS.caSecret.name, and coderTLS.clientSecret.name all use name, but listenerTLS.secretName uses secretName for the same concept (the name of a Kubernetes Secret). listenerTLS.secretName does mirror the coder chart's coder.tls.secretName, so the outlier has a cross-chart rationale, but the block is internally inconsistent. additionalProperties: false caps the harm (a wrong guess fails loudly). Cheapest to settle now, before this new chart's values API stabilizes. (ingress.tls.secretName mirrors the native Ingress field and should stay.)
🤖
| secretName: "" | ||
| certKey: tls.crt | ||
| keyKey: tls.key | ||
| terminationGracePeriodSeconds: 330 |
There was a problem hiding this comment.
Nit [CRF-31] terminationGracePeriodSeconds: 330 is coupled to the binary's shutdown timeout with nothing recording the coupling. (Gon)
CRF-19 raised that the previous value (300) equaled the binary's 5m shutdown timeout, so SIGKILL cut in-flight streams; the fix bumped it to 330 for headroom, which is correct. But no comment states that 330 is "binary shutdown timeout (~300s) plus headroom." A maintainer tuning this later sees only 330, cannot know it is anchored to an external timeout, and can lower it below the shutdown timeout, silently reintroducing CRF-19. Add a one-line comment tying the number to the binary's shutdown timeout. (Gon rated this P3; recorded as Nit since the consequence is regressing a Note-level issue and the fix is a comment.)
🤖
| 1. An AI Gateway key created in Coder. | ||
| 2. A Kubernetes Secret containing that key in the chart namespace: | ||
|
|
||
| ```console |
There was a problem hiding this comment.
Nit [CRF-32] The prerequisite says the Secret must live "in the chart namespace," but the kubectl create secret command has no -n. (Leorio)
README.md:12 states the Secret must be "in the chart namespace," then the command runs kubectl create secret generic coder-ai-gateway-key ... with no namespace flag, and helm install later also has no -n. On the happy path (current context namespace) this works. An operator who installs the release into a dedicated namespace but runs the secret command verbatim creates the Secret in default; the pod then fails to mount secretName: coder-ai-gateway-key and never starts, with only a Kubernetes mount error to debug. Add -n <release-namespace> to the command, or state that the Secret and the release must share a namespace.
🤖
| port: http | ||
| scheme: {{ $scheme }} | ||
| initialDelaySeconds: {{ .Values.coder.startupProbe.initialDelaySeconds }} | ||
| {{- range $field := list "periodSeconds" "timeoutSeconds" "successThreshold" "failureThreshold" }} |
There was a problem hiding this comment.
Nit [CRF-34] The probe field-projection block is triplicated verbatim across startupProbe, livenessProbe, and readinessProbe. (Meruem Nit, Robin Note)
The same range $field := list "periodSeconds" "timeoutSeconds" "successThreshold" "failureThreshold" / hasKey loop is copied three times (lines 106-110, 119-123, 132-136), differing only in the probe key and path. A define "coder-ai-gateway.probe" taking (probeName, path, scheme) would collapse the three and remove the drift risk when a field is added. The sibling coder chart duplicates the same logic (and less cleanly), so this extends a pre-existing pattern rather than introducing one; purely a maintainability nit.
🤖

This PR introduces a new
coder-ai-gateway Helm chart.It deploys the Coder AI Gateway as a standalone Kubernetes workload, and wires it into the existing build and release pipelines.
New
helm/ai-gatewaychart:Deploymentusing the sharedlibcoderlibrary chartSecretmounted at/etc/coder/ai-gateway-auth/keyaigateway.coderURLor a constructed in-cluster Service URL fromaigateway.coderService4001and a Prometheus metrics endpoint on port2112aigateway.listenerTLS) and Coder-facing mTLS/CA (aigateway.coderTLS)Service(ClusterIP/NodePort/LoadBalancer),Ingress, and Gateway APIHTTPRouteresourcescoder.envfrom overriding chart-owned environment variablesextraTemplatesfor additional manifests rendered within the releaseRelease pipeline integration:
build/ai-gateway_helm_<version>.tgzis now signed and uploaded as a release artifact in bothrelease.yamlandtag-and-release.yamlgs://helm.coder.com/v2, merged into the Helm repo index, and pushed tooci://ghcr.io/coder/chartBuild and tooling:
scripts/helm.shnow acceptsai-gatewayas a valid--chartvalueMakefileincludesai-gatewayin thechartslist, golden file generation, and golden file cleanup targetshelm/Makefileadds alint/helm/ai-gatewaytarget with the required lint flagsTests: