Lets Architect the Performance Engineering

You ramp traffic onto an LLM serving deployment and the graph does not add up. Time-to-first-token, flat at 300ms all morning, starts climbing: 900ms, 2s, 5s. Users are waiting. And the HorizontalPodAutoscaler that is supposed to save you is sitting at 2 replicas, perfectly calm, because the metric it watches — CPU — reads 22%. Nothing is broken, exactly. The autoscaler is doing precisely what you told it to do. You just told it to watch the wrong number. The graph that doesn't add up Here is the HPA's view of the world during the incident: $ kubectl describe hpa vllm-llama Name: vllm-llama Reference: Deployment/vllm-llama Metrics: ( current / target ) resource cpu on pods (as a percentage of request): 22% (110m) / 70% Min replicas: 2 Max replicas: 12 Deployment pods: 2 current / 2 desired Conditions: Type Status Reason Message ScalingActive True ValidMetricFound the HPA was able to compute a target ...

The green cutover with a red p99 The migration goes suspiciously well. An agent reads your EKS cluster, translates every manifest, generates the Terraform for the GKE side, and opens a change for review. You apply it. Every Deployment reports Ready , every synthetic check is green, the smoke tests pass. You cut traffic over. Then real traffic arrives and p99 latency climbs. Not by a rounding error — it roughly doubles on a couple of services and stays there. Nothing is crashing. No pod is CrashLoopBackOff . The dashboards the agent knew how to reason about are all green. The problem is that a Kubernetes manifest describes intent, and the agent proved manifest-equivalence, not behavior-equivalence. Those are different claims, and the gap between them is exactly the part of a migration that only shows up under load. What the agent is genuinely good at This is not a "don't use the tool" post. Agentic migration tooling — GKE's own EKS-to-GKE assistant, or a home-g...