perf(graph): warm schema cache on startup to kill cold-start spikes #843

Merged

argoyle merged 1 commits from warm-schema-cache-on-startup into main 2026-05-21 15:25:51 +00:00

Conversation 0 Commits 1 Files Changed 3

+85

1 Commits

Author	SHA1	Message	Date
argoyle	1549538c70	perf(graph): warm schema cache on startup to kill cold-start spikes ... Following the schema cache PR, warm pods serve from cache (~24/25 hits on a long-running pod). New pods, however, start cold: the first LatestSchema query per (orgId, ref) still runs the wgc router compose subprocess, which costs 100-300m CPU per call. That cold-start cost is what kept tripping the HPA into TooManyReplicas: HPA scales up → new pod added → new pod runs wgc on first query → metrics spike → HPA scales up further → cycle repeats. Even after the caching PR landed, observed pods cycling 2→4→2→4 in production, with fresh pods showing 2 'Fetching latest schema' (cold) entries and 0 cache hits within their first minute. Add Cache.AllOrgRefs() exposing every tracked (orgId, ref) pair, and Resolver.WarmCache(ctx) which iterates them after the event-sourced caches have been populated. For each ref it fetches the subgraphs, runs sdlmerge, runs CosmoGenerator.Generate, and stores both results in the cache. Errors per ref are logged and skipped so a single bad ref does not block warming the rest. Service startup calls WarmCache right after the Resolver is wired, before the HTTP server starts accepting traffic, so the first LatestSchema query a pod receives is already a cache hit.	2026-05-21 17:10:30 +02:00