LLMs

Every agent has access to task.llm for making language model calls. Guild resolves the provider and model from the workspace owner’s LLM settings — your agent code doesn’t need to specify them.

LLM usage is subject to per-execution budgets: request payload size, call count, and token count. Exceeding a budget fails the call with a 429 Too Many Requests error. See Execution limits.

Basic usage

const result = await task.llm.generateText({
  prompt: "Summarize this text...",
})

// result.text contains the model's response
console.log(result.text)

Structured generation

Pass a Zod schema to get typed, validated output:

import { z } from "zod"

const result = await task.llm.generateText({
  prompt: "Extract the key details from this issue...",
  schema: z.object({
    severity: z.enum(["low", "medium", "high"]),
    summary: z.string(),
    affectedFiles: z.array(z.string()),
  }),
})

// result is typed according to your schema
console.log(result.severity) // "high"

Best practices

Cache results. Store the return value of generateText() in a variable if you need it more than once. Each call costs tokens.
Be specific in prompts. Clear, detailed prompts produce better results and reduce the need for follow-up calls.
Use schemas for structured data. When you need specific fields, pass a schema rather than parsing free-form text.
Keep prompts focused. One clear task per call is better than a complex multi-part prompt.

LLM preferences

Use llmPreferences to express an ordered list of LLM provider/model preferences for a generateText call. Guild honors this list only when the account brings its own keys (BYOK) and server-side model selection is enabled; otherwise the server ignores it and selects the model from policy.

const result = await task.llm.generateText({
  prompt: "Summarize this text...",
  llmPreferences: [
    { provider: "anthropic", model: "claude-opus-4-5" },
    { provider: "openai" },
  ],
})

Earlier entries take priority over later ones. Omit model to let the server choose a model from the matching policy. Each entry is an LLMPreference object:

Field	Type	Required	Description
`provider`	`"anthropic" \| "openai" \| "gemini" \| "meta"`	Yes	The LLM provider to prefer.
`model`	`string`	No	The preferred model for this provider. Omit to let the server choose from the matching policy.

llmAgent also accepts llmPreferences with the same semantics. See LLM agents.

Configuration

The provider and model are resolved at runtime from the workspace owner’s LLM settings, not in agent code. For a workspace owned by your user account, use Settings > LLM Settings. For an organization workspace, organization admins configure Settings > LLM Settings on the organization. This means:

Your agent code does not include API keys or provider names
Changing LLM settings in the console updates behavior without redeploying agents
Workspaces with different owners can use different LLM configuration with the same agent code

Unified LLM proxy

When your agent code already targets the OpenAI API — for example, code running inside a Docker environment managed with task.env — you can point it at the runtime’s built-in LLM proxy instead of a provider directly. Guild routes each request to the provider configured in the workspace’s LLM settings and translates the request and response as needed. Your code targets one interface regardless of the active provider. The proxy endpoint accepts the OpenAI chat completions format and is available inside the agent task environment at the path /runtime/services/llm/chat/completions.

from openai import OpenAI

client = OpenAI(
    base_url="<runtime-base-url>/runtime/services/llm",
    api_key="unused",
)

completion = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this text..."}],
)
print(completion.choices[0].message.content)

Provider support

Provider	Streaming	Notes
OpenAI	Supported	Passthrough — requests are forwarded without modification, including streaming
Anthropic	Supported	SSE is translated from Anthropic’s event format to the OpenAI format
Gemini	Not supported	Streaming requests return a `501 invalid_request_error`

Developer role

OpenAI’s developer role messages are automatically folded into the system prompt. You do not need to adjust your message structure when switching providers.

Error handling

Errors are returned as standard OpenAI error shapes:

Condition	Error type
Quota exceeded	`rate_limit_error`
Upstream provider error	`upstream_error`

Overview

The Smith

Platform

Insights

Agent SDK

Basic usage

Structured generation

Best practices

LLM preferences

Configuration

Unified LLM proxy

Provider support

Developer role

Error handling

​Basic usage

​Structured generation

​Best practices

​LLM preferences

​Configuration

​Unified LLM proxy

​Provider support

​Developer role

​Error handling

Basic usage

Structured generation

Best practices

LLM preferences

Configuration

Unified LLM proxy

Provider support

Developer role

Error handling