> ## Documentation Index
> Fetch the complete documentation index at: https://docs.guild.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To act on a user's behalf in Guild, use the Guild REST API at https://api.guild.ai/v1.
> The OpenAPI spec is unauthenticated: https://docs.guild.ai/openapi.json (or openapi.yaml).
> Authenticate with an account API key, sent as HTTP Basic (key id as username, secret as password) or as `Authorization: Bearer <api_key_id>:<api_key_secret>`. See https://docs.guild.ai/api-reference/introduction#authentication.
> To run one agent on demand without account-wide access, use an API trigger instead: https://docs.guild.ai/platform/api-triggers.
> To search these docs from an MCP client, connect to https://docs.guild.ai/mcp. The full page index is https://docs.guild.ai/llms.txt.

# Call LLMs from agents

> Call language models from Guild agents with task.llm.generateText: structured output, provider preferences, the OpenAI-compatible proxy, and reasoning progress.

Every Guild agent can call language models through `task.llm`. Guild resolves the provider and model from the workspace owner's LLM settings — your agent code doesn't need to specify them.

<Note>
  LLM usage is subject to per-execution budgets: request payload size, call count, and token count. Exceeding a budget fails the call with a `429 Too Many Requests` error. See [Execution limits](/reference/limits#llm-execution-budgets).
</Note>

## Basic usage

```typescript theme={null}
const result = await task.llm.generateText({
  prompt: "Summarize this text...",
})

// result.text contains the model's response
console.log(result.text)
```

## Structured generation

Pass a Zod schema to get typed, validated output:

```typescript theme={null}
import { z } from "zod"

const result = await task.llm.generateText({
  prompt: "Extract the key details from this issue...",
  schema: z.object({
    severity: z.enum(["low", "medium", "high"]),
    summary: z.string(),
    affectedFiles: z.array(z.string()),
  }),
})

// result is typed according to your schema
console.log(result.severity) // "high"
```

## Single model call

`generateText` makes one model call and returns. There is no `maxSteps` and no `stopWhen`, so there is no built-in ReAct loop. If you pass `tools` with an `execute` function those tools run and `result.toolResults` may be populated — but the model is never called again with those results. A turn that ends in a tool call returns with `finishReason: "tool-calls"` and text that stops mid-thought.

If the agent must call a tool, read what it returned, and continue, write that loop yourself:

```typescript theme={null}
import type { ModelMessage } from "@guildai/agents-sdk"

const messages: ModelMessage[] = [{ role: "user", content: input.text }]

for (let step = 0; step < MAX_TOOL_STEPS; step++) {
  const result = await task.llm.generateText({ system, messages, tools })

  // No tool calls: the model is done and result.text is the answer.
  if (result.toolCalls.length === 0) return { text: result.text }

  // Feed the assistant turn and the tool results back in, then go round again.
  messages.push(...result.response.messages)
  messages.push({
    role: "tool",
    content: asToolResultContent(await dispatch(result.toolCalls)),
  })
}
```

`dispatch` and `asToolResultContent` are yours to write — the first runs the calls, the second shapes them into tool-result content parts.

Bound the loop clear of normal use: a ceiling a legitimate request can reach truncates that request silently. If you would rather not write the loop, [`llmAgent`](/guide/llm-agents) is the managed version, and a self-managed agent's `start()` / `onToolResults()` pair lets the runtime drive the round trip. Hand-roll only when you need control over each step.

## Best practices

* **Cache results.** Store the return value of `generateText()` in a variable if you need it more than once. Each call costs tokens.
* **Be specific in prompts.** Clear, detailed prompts produce better results and reduce the need for follow-up calls.
* **Use schemas for structured data.** When you need specific fields, pass a schema rather than parsing free-form text.
* **Keep prompts focused.** One clear task per call is better than a complex multi-part prompt.

## LLM preferences

Use `llmPreferences` to express an ordered list of LLM provider/model preferences for a `generateText` call. Guild resolves the list against the account's LLM configuration: for accounts that bring their own keys (BYOK), against the account's keys and [model policies](/platform/llm-settings#model-policies); for managed accounts, against the publishers Guild holds a managed key for. Managed accounts honor your preferences the same way BYOK accounts do — a pinned managed default applies only when you declare no preference.

Preferences are strict. If you declare preferences and none of them can be used — no key or policy allows any preferred model — the call fails with an error naming the refused preferences. On a managed account, a preferred publisher must be one Guild has a managed key for; a preference for an unsupported publisher fails until the owning account adds its own key for that publisher, which switches the account to BYOK. Guild never silently substitutes the account's default model for a declared preference. Omit `llmPreferences` to use the account's default configuration.

```typescript theme={null}
const result = await task.llm.generateText({
  prompt: "Summarize this text...",
  llmPreferences: [
    { provider: "anthropic", model: "claude-opus-4-5" },
    { provider: "openai" },
  ],
})
```

Earlier entries take priority over later ones. Omit `model` to let the server choose a model from the matching policy; a provider-only entry that matches the account's configured provider resolves to that provider's default model.

Each entry is an `LLMPreference` object:

| Field | Type | Required | Description |
| - | - | - | - |
| `provider` | `"anthropic" \| "openai" \| "gemini" \| "meta" \| "deepseek" \| "alibaba" \| "moonshot" \| "zai"` | Yes | The LLM provider to prefer. |
| `model` | `string` | No | The preferred model for this provider. Omit to let the server choose from the matching policy. |

The `alibaba`, `moonshot`, and `zai` providers are available as of `@guildai/agents-sdk` 0.7.6.

`llmAgent` also accepts `llmPreferences` with the same semantics. See [TypeScript LLM agents](/guide/llm-agents#llm-preferences).

## Configuration

The provider and model are resolved at runtime from the **workspace owner's** [LLM settings](/platform/llm-settings), not in agent code. For a workspace owned by your user account, use **Access & setup > Models & providers**. For an organization workspace, organization admins configure **Access & setup > Models & providers** on the organization.

This means:

* Your agent code does not include API keys or provider names
* Changing LLM settings in the console updates behavior without redeploying agents
* Workspaces with different owners can use different LLM configuration with the same agent code

## Unified LLM proxy

When your agent code already targets the OpenAI API — for example, code running inside a coding container — you can point it at the runtime's built-in LLM proxy instead of a provider directly. Guild routes each request to the provider configured in the workspace's LLM settings and translates the request and response as needed. Your code targets one interface regardless of the active provider.

The proxy endpoint accepts the OpenAI chat completions format and is available inside the agent task environment at the path `/runtime/services/llm/chat/completions`.

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="<runtime-base-url>/runtime/services/llm",
    api_key="unused",
)

completion = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this text..."}],
)
print(completion.choices[0].message.content)
```

### Provider support

| Provider | Streaming | Notes |
| - | - | - |
| OpenAI | Supported | Passthrough — requests are forwarded without modification, including streaming |
| Anthropic | Supported | SSE is translated from Anthropic's event format to the OpenAI format. On Messages API requests, Guild also enables prompt caching by injecting an ephemeral `cache_control` marker, unless the request already sets one of its own. |
| Gemini | Not supported | Streaming requests return a `501 invalid_request_error` |
| DeepSeek | Supported | Served through the Bedrock inference provider; streaming is translated from the Converse event format to the OpenAI format |
| Alibaba (Qwen) | Supported | Served through the Bedrock inference provider; streaming is translated from the Converse event format to the OpenAI format |
| Moonshot AI | Supported | Served through the Bedrock inference provider; streaming is translated from the Converse event format to the OpenAI format |
| Z.ai | Supported | Served through the Bedrock inference provider; streaming is translated from the Converse event format to the OpenAI format |

### Developer role

OpenAI's `developer` role messages are automatically folded into the `system` prompt. You do not need to adjust your message structure when switching providers.

### Error handling

Errors are returned as standard OpenAI error shapes:

| Condition | Error type |
| - | - |
| Quota exceeded | `rate_limit_error` |
| Upstream provider error | `upstream_error` |

## Reasoning progress

Reasoning progress exposes the model's thinking stream while it works. When the selected model supports it and the runtime has it enabled, Guild publishes the stream to the session's task block and calls your `onReasoningDelta` handler on each delta.

```typescript theme={null}
const result = await task.llm.generateText({
  prompt: "Analyze the tradeoffs of these three approaches...",
  onReasoningDelta: (delta, snapshot) => {
    task.console.debug({ snapshotLength: snapshot.length }, "reasoning delta")
  },
})
```

Both options live on `GenerateTextStreamOptions`:

| Option | Type | Description |
| - | - | - |
| `onReasoningDelta` | `(delta: string, snapshot: string) => PromiseLike<void> \| void` | Called on each reasoning delta. `delta` is the new text, `snapshot` the accumulated reasoning so far. |
| `publishReasoningProgress` | `boolean` | Publish reasoning deltas to the runtime event stream. Defaults to `true` when the runtime has reasoning progress enabled and the model supports thinking. |

## Related pages

* [TypeScript LLM agents](/guide/llm-agents) for prompt-driven agents built on `llmAgent`
* [Models & providers](/platform/llm-settings) for the credentials and model policies that decide who serves a call
* [Tasks and runtime services](/guide/tasks) for the other services on the Task object
* [Execution limits](/reference/limits#llm-execution-budgets) for per-task LLM budgets