LLMs

Every agent has access to task.llm for making language model calls. Guild resolves the provider and model from the workspace owner’s LLM settings — your agent code doesn’t need to specify them.

LLM usage is subject to per-execution budgets: request payload size, call count, and token count. Exceeding a budget fails the call with a 429 Too Many Requests error. See Execution limits.

Basic usage

const result = await task.llm.generateText({
  prompt: "Summarize this text...",
})

// result.text contains the model's response
console.log(result.text)

Structured generation

Pass a Zod schema to get typed, validated output:

import { z } from "zod"

const result = await task.llm.generateText({
  prompt: "Extract the key details from this issue...",
  schema: z.object({
    severity: z.enum(["low", "medium", "high"]),
    summary: z.string(),
    affectedFiles: z.array(z.string()),
  }),
})

// result is typed according to your schema
console.log(result.severity) // "high"

Single model call

generateText makes one model call and returns. There is no maxSteps and no stopWhen, so there is no built-in ReAct loop. If you pass tools with an execute function those tools run and result.toolResults may be populated — but the model is never called again with those results. A turn that ends in a tool call returns with finishReason: "tool-calls" and text that stops mid-thought. If the agent must call a tool, read what it returned, and continue, write that loop yourself:

import type { ModelMessage } from "@guildai/agents-sdk"

const messages: ModelMessage[] = [{ role: "user", content: input.text }]

for (let step = 0; step < MAX_TOOL_STEPS; step++) {
  const result = await task.llm.generateText({ system, messages, tools })

  // No tool calls: the model is done and result.text is the answer.
  if (result.toolCalls.length === 0) return { text: result.text }

  // Feed the assistant turn and the tool results back in, then go round again.
  messages.push(...result.response.messages)
  messages.push({
    role: "tool",
    content: asToolResultContent(await dispatch(result.toolCalls)),
  })
}

dispatch and asToolResultContent are yours to write — the first runs the calls, the second shapes them into tool-result content parts. Bound the loop clear of normal use: a ceiling a legitimate request can reach truncates that request silently. If you would rather not write the loop, llmAgent is the managed version, and a self-managed agent’s start() / onToolResults() pair lets the runtime drive the round trip. Hand-roll only when you need control over each step.

Best practices

Cache results. Store the return value of generateText() in a variable if you need it more than once. Each call costs tokens.
Be specific in prompts. Clear, detailed prompts produce better results and reduce the need for follow-up calls.
Use schemas for structured data. When you need specific fields, pass a schema rather than parsing free-form text.
Keep prompts focused. One clear task per call is better than a complex multi-part prompt.

LLM preferences

Use llmPreferences to express an ordered list of LLM provider/model preferences for a generateText call. Guild resolves the list against the account’s LLM configuration: for accounts that bring their own keys (BYOK), against the account’s keys and model policies; for managed accounts, against server-side authorizations. Preferences are strict. If you declare preferences and none of them can be used — no key or policy allows any preferred model — the call fails with an error naming the refused preferences. Guild never silently substitutes the account’s default model for a declared preference. Omit llmPreferences to use the account’s default configuration.

const result = await task.llm.generateText({
  prompt: "Summarize this text...",
  llmPreferences: [
    { provider: "anthropic", model: "claude-opus-4-5" },
    { provider: "openai" },
  ],
})

Earlier entries take priority over later ones. Omit model to let the server choose a model from the matching policy; a provider-only entry that matches the account’s configured provider resolves to that provider’s default model. Each entry is an LLMPreference object:

Field	Type	Required	Description
`provider`	`"anthropic" \| "openai" \| "gemini" \| "meta"`	Yes	The LLM provider to prefer.
`model`	`string`	No	The preferred model for this provider. Omit to let the server choose from the matching policy.

llmAgent also accepts llmPreferences with the same semantics. See TypeScript LLM agents.

Configuration

The provider and model are resolved at runtime from the workspace owner’s LLM settings, not in agent code. For a workspace owned by your user account, use Settings > LLM Settings. For an organization workspace, organization admins configure Settings > LLM Settings on the organization. This means:

Your agent code does not include API keys or provider names
Changing LLM settings in the console updates behavior without redeploying agents
Workspaces with different owners can use different LLM configuration with the same agent code

Unified LLM proxy

When your agent code already targets the OpenAI API — for example, code running inside a coding container — you can point it at the runtime’s built-in LLM proxy instead of a provider directly. Guild routes each request to the provider configured in the workspace’s LLM settings and translates the request and response as needed. Your code targets one interface regardless of the active provider. The proxy endpoint accepts the OpenAI chat completions format and is available inside the agent task environment at the path /runtime/services/llm/chat/completions.

from openai import OpenAI

client = OpenAI(
    base_url="<runtime-base-url>/runtime/services/llm",
    api_key="unused",
)

completion = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this text..."}],
)
print(completion.choices[0].message.content)

Provider support

Provider	Streaming	Notes
OpenAI	Supported	Passthrough — requests are forwarded without modification, including streaming
Anthropic	Supported	SSE is translated from Anthropic’s event format to the OpenAI format. On Messages API requests, Guild also enables prompt caching by injecting an ephemeral `cache_control` marker, unless the request already sets one of its own.
Gemini	Not supported	Streaming requests return a `501 invalid_request_error`

Developer role

OpenAI’s developer role messages are automatically folded into the system prompt. You do not need to adjust your message structure when switching providers.

Error handling

Errors are returned as standard OpenAI error shapes:

Condition	Error type
Quota exceeded	`rate_limit_error`
Upstream provider error	`upstream_error`

Reasoning progress

Reasoning progress exposes the model’s thinking stream while it works. When the selected model supports it and the runtime has it enabled, Guild publishes the stream to the session’s task block and calls your onReasoningDelta handler on each delta.

const result = await task.llm.generateText({
  prompt: "Analyze the tradeoffs of these three approaches...",
  onReasoningDelta: (delta, snapshot) => {
    task.console.debug({ snapshotLength: snapshot.length }, "reasoning delta")
  },
})

Both options live on GenerateTextStreamOptions:

Option	Type	Description
`onReasoningDelta`	`(delta: string, snapshot: string) => PromiseLike<void> \| void`	Called on each reasoning delta. `delta` is the new text, `snapshot` the accumulated reasoning so far.
`publishReasoningProgress`	`boolean`	Publish reasoning deltas to the runtime event stream. Defaults to `true` when the runtime has reasoning progress enabled and the model supports thinking.

Overview

The Smith

The Factory

Platform

Insights

Agent SDK

Basic usage

Structured generation

Single model call

Best practices

LLM preferences

Configuration

Unified LLM proxy

Provider support

Developer role

Error handling

Reasoning progress

​Basic usage

​Structured generation

​Single model call

​Best practices

​LLM preferences

​Configuration

​Unified LLM proxy

​Provider support

​Developer role

​Error handling

​Reasoning progress

Basic usage

Structured generation

Single model call

Best practices

LLM preferences

Configuration

Unified LLM proxy

Provider support

Developer role

Error handling

Reasoning progress