Lets Architect the Performance Engineering

An agent answers a question that requires it to run a three-line Python snippet — parse a CSV, sum a column, return the total. The code runs in about 300 milliseconds. The user waits eight seconds. Trace the request and the actual computation is a rounding error; everything else is the agent waiting for somewhere to run the code. The bill tells the same story from the other side. You provisioned a generously sized pool of execution environments so agents never queue, and the utilization dashboard shows them sitting idle most of the day, warm and billable, waiting for the next tool call. You are paying full compute prices to keep empty rooms lit. Both symptoms — the latency and the cost — come from the same design decision: one fresh, full-sized sandbox per agent, cold on the way in and idle on the way out. Where the eight seconds go An agent that executes code or calls tools needs an isolated environment to do it in — you cannot run model-generated code in ...