fix(k8s): add scaleUp/scaleDown stabilization to schemas HPA #844

Merged

argoyle merged 1 commits from tune-hpa-behavior into main 2026-05-21 17:10:19 +00:00

Conversation 0 Commits 1 Files Changed 1

+20

argoyle commented 2026-05-21 16:56:23 +00:00

Owner

Copy Link

Copy Source

Even with the schema cache (#841) and startup warmup (#843), residual CPU bursts (metrics-server occasionally samples a request mid-flight) were enough to trip a brief scale-up, after which the default 5min scaleDown stabilization pinned the deployment at maxReplicas long after the spike had subsided.

Tune both directions

• scaleUp.stabilizationWindowSeconds: 120 — a transient spike must persist for two consecutive minutes before any pod is added. Brief metric anomalies no longer move replicas.
• scaleUp policy: add at most 1 pod per 60s. Smooths reaction.
• scaleDown.stabilizationWindowSeconds: 120 (default 300) — once the workload calms, return to minReplicas faster.
• scaleDown policy: remove at most 1 pod per 60s. Avoids thundering-herd scale-down.

Net effect: replicas now spend most of the time at minReplicas (2) and only grow under sustained load.

Even with the schema cache (#841) and startup warmup (#843), residual CPU bursts (metrics-server occasionally samples a request mid-flight) were enough to trip a brief scale-up, after which the default 5min scaleDown stabilization pinned the deployment at maxReplicas long after the spike had subsided. ## Tune both directions - `scaleUp.stabilizationWindowSeconds: 120` — a transient spike must persist for two consecutive minutes before any pod is added. Brief metric anomalies no longer move replicas. - `scaleUp` policy: add at most 1 pod per 60s. Smooths reaction. - `scaleDown.stabilizationWindowSeconds: 120` (default 300) — once the workload calms, return to minReplicas faster. - `scaleDown` policy: remove at most 1 pod per 60s. Avoids thundering-herd scale-down. Net effect: replicas now spend most of the time at minReplicas (2) and only grow under sustained load.

argoyle added 1 commit 2026-05-21 16:56:25 +00:00

fix(k8s): add scaleUp/scaleDown stabilization to schemas HPA ... eefab471d6

Even with the schema cache and startup warmup, residual CPU bursts
(metrics-server occasionally samples a request mid-flight) were enough
to trip a brief scale-up, after which the default 5min scaleDown
stabilization pinned the deployment at maxReplicas long after the
spike had subsided.

Tune both directions:

- scaleUp.stabilizationWindowSeconds: 120 — a transient spike must
  persist for two consecutive minutes before any pod is added.
  Brief metric anomalies no longer move replicas.
- scaleUp policy: add at most 1 pod per 60s. Smooths reaction.
- scaleDown.stabilizationWindowSeconds: 120 (default 300) — once
  the workload calms, return to minReplicas faster.
- scaleDown policy: remove at most 1 pod per 60s. Avoids
  thundering-herd scale-down.

argoyle scheduled this pull request to auto merge when all checks succeed 2026-05-21 16:56:38 +00:00

argoyle merged commit 111c2e4b19 into main 2026-05-21 17:10:19 +00:00

argoyle deleted branch tune-hpa-behavior 2026-05-21 17:10:20 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

dependencies

Pull requests that update a dependency file

docker

go

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

acctest (Shiny Acctest) argoyle (Joakim Olsson) buildtools (Buildtools) kubernetes (Kubernetes) releaser (Unbound Releaser) renovate (Renovate Bot)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unboundsoftware/schemas#844