Optimizations is currently being offered to a select group of customers. If Optimizations doesn’t appear in your sidebar, the feature isn’t enabled for your account yet.
Starting a run
Open Optimizations in the sidebar, or click Optimize next to a workspace in the Workspaces breakdown on the Insights page. A workspace can only be optimized if it’s eligible:- One agent in the workspace — the entrypoint — must be owned by your account, still installed in the workspace, and have at least 10 recent sessions. Guild picks the workspace agent with the most recent activity as the entrypoint automatically.
- At least one agent in the workspace (the entrypoint or a sub-agent it calls) must actually appear in those sessions, so there’s something to measure.
- The workspace can’t already have a run in progress. Only one optimization run is active per workspace at a time.
How a run works
Building a baseline evaluation
Optimizations replays an agent’s own history to judge whether a cheaper configuration is still good enough, so the first step is turning that history into an evaluation. Guild takes the entrypoint’s most recent sessions (10 by default) and reduces each one to what actually happened — the triggering input, the tool calls made, and the final outcome — rather than the raw conversation. An LLM turns each of those digests into an evaluation sample: the original trigger becomes the input, and the outcome becomes what a judge checks the optimized agent against. Before that evaluation is used to compare anything, Guild runs the agent’s current, unmodified configuration against it. Since every sample is drawn from a session the agent already completed successfully, the current configuration is expected to pass. If it doesn’t, Guild diagnoses why (a missing fact, a reference that’s too strict, or a sample that can’t be reproduced) and repairs the evaluation, for up to two rounds, before measuring any candidates against it. You can view the generated evaluation from the run screen. Editing its samples isn’t available from the Optimizations UI today.Testing cheaper configurations
Once the baseline evaluation holds up, Guild tests configuration changes — called candidates — against it:
Each candidate is actually built and run against the same evaluation as the baseline — Guild doesn’t estimate savings, it measures them. A candidate only counts as a real result once enough of its trials complete and enough samples score on both the baseline and the candidate to compare fairly.
Scoring the result
Every candidate gets a verdict based on how its quality compares to the baseline:- Positive — quality held (within a small tolerance) or improved.
- Neutral — quality dropped slightly, within a wider tolerance.
- Negative — quality dropped meaningfully, or the candidate lost tool calls or checks the baseline passed.
Reviewing and accepting changes
The run screen lists every agent Guild measured, with its candidates side by side — baseline, model only, tool trim, prompt rewrite, and combined — showing the quality change and cost change for each. A candidate that lost a tool call or check the baseline passed is marked Behavior changed regardless of its cost. For each agent with a recommended change, you can:- Accept — installs the recommended configuration for that agent. Guild publishes it as a new agent version, points the workspace at that version, and turns off autoupdate for the agent so a later publish doesn’t silently overwrite the optimization. This can take a few minutes and can’t be stopped once it starts.
- Accept all — accepts every agent in the run with a recommended change in one step.
- Dismiss — declines the recommended change for that agent. This can’t be undone from the run screen.
- Cancel — stops the whole run. Any agent still awaiting a decision is dismissed.