# Use another model: Gemini, Claude, Grok, DeepSeek or a local one, in the same harness (/docs/agent/use-another-model)

---
type: Procedure
title: "Use another model: Gemini, Claude, Grok, DeepSeek or a local one, in the same harness"
description: "The book's two ways to switch the model under the OpenAI Agents SDK, a base-URL swap for any OpenAI-compatible endpoint and LiteLLM for everything else, with the key check, the tracing caveat, the economy-versus-frontier split per agent, and what was verified on 2026-09-04: Gemini through both paths with a tool call, and a local llama.cpp model through the base-URL path; and on 2026-10-12, Claude through Anthropic's OpenAI-compatible endpoint and LiteLLM and Grok through xAI's Responses and Chat Completions endpoints and LiteLLM, each route reaching the provider, plus Gemini's free-tier gemini-3.1-flash-lite through both paths with a tool call."
status: stable
order: 20
generated: { by: "process:claude-code", at: 2026-10-12T07:57:06Z }
verified:
  - { by: "human:rrizwan98", at: 2026-09-04T10:50:00Z }
  - { by: "human:rrizwan98", at: 2026-10-12T07:57:06Z }
stale_after: 2027-01-15T00:00:00Z
sources:
  - id: af-39
    title: "Build AI Agents with the OpenAI Agents SDK: A 90-Minute Crash Course, Concept 12 'Switching models', 'Reaching providers that aren't OpenAI-compatible: LiteLLM', the API key format check, and Part 6 'Cost discipline: routing by model tier'. The AI Agent Factory, Panaversity, 2026"
    resource: https://agentfactory.panaversity.org/docs/build-agents-crash-course
  - id: sdk-models
    title: "Models: using non-OpenAI models, LiteLLM adapter, OpenAI-compatible endpoints, tracing and common issues. OpenAI Agents SDK documentation"
    resource: https://openai.github.io/openai-agents-python/models/
  - id: gemini-openai
    title: "OpenAI compatibility. Gemini API documentation, Google"
    resource: https://ai.google.dev/gemini-api/docs/openai
  - id: llama-cpp
    title: "llama.cpp release b10797, Windows CPU build with llama-server (an OpenAI-compatible local server). ggml-org, GitHub"
    resource: https://github.com/ggml-org/llama.cpp/releases/tag/b10797
  - id: pypi-openai-agents
    title: "openai-agents 0.22.0 on PyPI, optional extras including litellm and any-llm"
    resource: https://pypi.org/project/openai-agents/
  - id: anthropic-openai
    title: "OpenAI SDK compatibility: base URL, supported fields and limitations. Claude API documentation, Anthropic, retrieved 2026-10-12"
    resource: https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk
  - id: xai-function-calling
    title: "Function calling: the OpenAI SDK pointed at the xAI API, XAI_API_KEY, tool_choice. xAI documentation, retrieved 2026-10-12"
    resource: https://docs.x.ai/docs/guides/function-calling
  - id: xai-comparison
    title: "Model capabilities: the Responses API recommended, Chat Completions labelled deprecated (legacy). xAI documentation, retrieved 2026-10-12"
    resource: https://docs.x.ai/docs/guides/chat/comparison
  - id: xai-models
    title: "Models and pricing: current Grok model ids. xAI documentation, retrieved 2026-10-12"
    resource: https://docs.x.ai/docs/models
  - id: raza-provider-check
    title: "Provider check under openai-agents 0.23.1, 12 October 2026: Claude, Grok and Gemini paths (attestation)"
    resource: "attestation: run by process:claude-code for Raza Rizwan on 2026-10-12; Gemini with the owner's free-tier key, Anthropic and xAI with a deliberately invalid key to prove each route reaches the provider without spending; not client work"
  - id: raza-mortgage-demo
    title: "Owner's review of a mortgage follow-up demo built from this record, 12 October 2026 (attestation)"
    resource: "attestation: Raza Rizwan, owner; review conducted with process:claude-code on 2026-10-12 of a demo build on invented pilot data; not client work"
ksor:
  audience: [public]
  owner: human:rrizwan98
  approval: { by: "human:rrizwan98", at: 2026-10-12T07:57:06Z }
trust_tier: human-reviewed
build_id: sha256:4d4bff09bd0a1dcff0a0a50f20f7ef514fc471e2b1f9e81e0e7b6150b93c0793
dirty: true
---

Almost nothing in the room is OpenAI-specific. Swap the model for DeepSeek,
or for Claude or Gemini through LiteLLM, and the tools, sessions, guardrails,
approval gate, harness, container and end-to-end checks stay exactly as they
are. [^af-39] This page is the swap, taken from the book's own concept and
verified against the installed SDK, so that a person who does not want to
use OpenAI, or wants a model on their own machine, can still run every other
page in this room.

## Two paths, one rule

The SDK supports any OpenAI-API-compatible model through a base URL plus API
key swap; for providers that only speak their own API, it ships LiteLLM as
an optional extra. The rule for which path to take: [^af-39]

| The provider gives you | Use |
| --- | --- |
| An OpenAI-compatible endpoint (DeepSeek, Groq, a local vLLM server) | The base-URL swap, no new dependency |
| Only its own native API (Claude, Gemini, Bedrock, Ollama) | `LitellmModel` and the `[litellm]` extra |

Since the book was written, the providers people ask for most all publish
an OpenAI-compatible endpoint as well, so both paths reach them; the
sections below on Claude and Grok say which path each provider recommends
and what was verified here.

The minimum change is not the model string and not a reinstall: it is a
client pointed at the right endpoint, and a typed model object passed where
a string went before. [^af-39]

## Path 1: the base-URL swap

The book's `models.py`, which defaults to OpenAI and swaps both tiers to
DeepSeek when a `DEEPSEEK_API_KEY` exists: [^af-39]

```python
# src/chat_agent/models.py
import os

from openai import AsyncOpenAI

from agents import OpenAIChatCompletionsModel

# NOTE: do not call set_tracing_disabled(True) here. The CLI decides per-run
# via RunConfig(tracing_disabled=...) based on whether an OPENAI_API_KEY is
# set. A global disable would silently shut off tracing even after a learner
# adds an OpenAI key later.

# Default to OpenAI on the standard client (the chapter's primary path).
# If DEEPSEEK_API_KEY is set, swap both models to the DeepSeek endpoint
# via the OpenAI-compatible client. Call sites stay identical either way:
# Agent(model=flash_model, ...) accepts a string or a typed model object.
flash_model: str | OpenAIChatCompletionsModel = "gpt-5.4-mini"
pro_model: str | OpenAIChatCompletionsModel = "gpt-5.5"

deepseek_key: str | None = os.environ.get("DEEPSEEK_API_KEY")
if deepseek_key:
    deepseek_client: AsyncOpenAI = AsyncOpenAI(
        api_key=deepseek_key,
        base_url="https://api.deepseek.com",
    )
    flash_model = OpenAIChatCompletionsModel(
        model="deepseek-v4-flash",
        openai_client=deepseek_client,
    )
    pro_model = OpenAIChatCompletionsModel(
        model="deepseek-v4-pro",
        openai_client=deepseek_client,
    )
```

Then pass the model object instead of a string anywhere you have
`Agent(...)`; everything else works identically. [^af-39] In this room that
is one line in the factory on the [build page](build-the-text-agent.md):
`Agent(name=..., instructions=..., tools=[...], model=flash_model)`.

The names change, the pattern does not: the book's economy model is today's
cheapest OpenAI-compatible one, and if it is not when you read this, search
for the current one and swap the model string. The mechanism the code
depends on is only the client and the base URL. [^af-39]

**The same swap reaches Gemini and a local model.** Google publishes an
OpenAI-compatible endpoint for the Gemini API, in beta while feature support
is extended, with function calling and structured output supported:
[^gemini-openai]

```python
# Gemini through its OpenAI-compatible endpoint (verified 2026-09-04)
gemini_client = AsyncOpenAI(
    api_key=os.environ["GEMINI_API_KEY"],
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)
flash_model = OpenAIChatCompletionsModel(model="gemini-2.5-flash", openai_client=gemini_client)
```

```python
# A model on your own machine, behind any OpenAI-compatible local server
# (verified 2026-09-04 with llama.cpp's llama-server on 127.0.0.1:8081)
local_client = AsyncOpenAI(api_key="local", base_url="http://127.0.0.1:8081/v1")
flash_model = OpenAIChatCompletionsModel(model="qwen2.5-0.5b-instruct", openai_client=local_client)
```

## Path 2: LiteLLM, for any model

Some models you will want do not offer an OpenAI-compatible endpoint at all;
LiteLLM puts Anthropic, Google, AWS Bedrock, Mistral, local Ollama and many
more behind one model object, as an optional extra: [^af-39]

```bash
uv add "openai-agents[litellm]"
```

Then construct a `LitellmModel` exactly where you constructed
`OpenAIChatCompletionsModel` before. The provider lives in the model string
as a `provider/model` prefix; the key is passed in directly: [^af-39]

```python
# src/chat_agent/models.py (the any-provider path)
import os

from agents.extensions.models.litellm_model import LitellmModel

# Claude, via Anthropic's native API:
claude_model = LitellmModel(
    model="anthropic/claude-4.5-sonnet",        # provider/model; verify the current id
    api_key=os.environ["ANTHROPIC_API_KEY"],
)

# Gemini, Bedrock, Ollama, and the rest follow the same shape:
# LitellmModel(model="gemini/...", api_key=os.environ["GEMINI_API_KEY"])
```

A `LitellmModel` is a model object, so the call site is unchanged; it drops
straight into `Agent(model=...)`. [^af-39] The SDK documentation adds that a
`litellm/...` model name string also works in place of the object, and that
some providers reached through LiteLLM do not populate usage metrics unless
you pass `ModelSettings(include_usage=True)`. [^sdk-models]

## Claude (Anthropic)

Anthropic publishes an OpenAI SDK compatibility layer: point the client at
`https://api.anthropic.com/v1/` with a Claude API key and a Claude model
name. Anthropic says the layer is primarily intended to test and compare
model capabilities, is not considered a long-term or production-ready
solution for most use cases, and recommends its native API for the full
feature set. [^anthropic-openai] So the two paths split by depth:

| Depth | Path | Why |
| --- | --- | --- |
| A [demo](demo-or-production.md) | The base-URL swap | No new dependency; the layer is meant for exactly this kind of trial |
| Production | `LitellmModel` with an `anthropic/` model string | LiteLLM speaks Anthropic's native API, which Anthropic recommends |

```python
# Claude, demo depth: Anthropic's OpenAI-compatible endpoint (route verified 2026-10-12)
claude_client = AsyncOpenAI(
    api_key=os.environ["ANTHROPIC_API_KEY"],
    base_url="https://api.anthropic.com/v1/",
)
flash_model = OpenAIChatCompletionsModel(model="claude-haiku-5-5", openai_client=claude_client)

# Claude, production depth: the native API through LiteLLM (route verified 2026-10-12)
from agents.extensions.models.litellm_model import LitellmModel

flash_model = LitellmModel(model="anthropic/claude-haiku-5-5", api_key=os.environ["ANTHROPIC_API_KEY"])
```

Four things the compatibility layer does differently, from Anthropic's own
table, each one a test to write before relying on it: [^anthropic-openai]

- **`strict` on a function tool is ignored**, so tool arguments are not
  guaranteed to match the schema; validate them in the tool body, which this
  room does anyway ([build page](build-the-text-agent.md)).
- **`response_format` is ignored**, so a typed output is not enforced by the
  provider; parse and validate the output in code.
- **`temperature` and `top_p`:** on Claude 4.7 and later models, a
  `temperature` below 1 or any `top_p` other than the default returns a 400
  error. Leave both out of `ModelSettings`.
- **Forced tool use:** `tool_choice="required"` or a named function returns
  an error on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1; keep
  `tool_choice` at `"auto"` on those models.

The model id is never guessed: list the account's models first, as the key
check below shows, and use an id the list returns.

## Grok (xAI)

xAI's API takes the OpenAI SDK with the base URL `https://api.x.ai/v1` and a
key read from `XAI_API_KEY`, and its function-calling examples use the
Responses API. [^xai-function-calling] xAI recommends the Responses API and
labels its Chat Completions endpoint deprecated, a legacy endpoint with
limited updates. [^xai-comparison] The Agents SDK speaks the Responses API
natively, so Grok takes `OpenAIResponsesModel` with the xAI client, and
`OpenAIChatCompletionsModel` only as a fallback:

```python
# Grok through xAI's Responses API (route verified 2026-10-12)
from agents import OpenAIResponsesModel

xai_client = AsyncOpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")
flash_model = OpenAIResponsesModel(model="grok-4.7", openai_client=xai_client)

# LiteLLM reaches the same provider with an xai/ prefix (route verified 2026-10-12)
flash_model = LitellmModel(model="xai/grok-4.7", api_key=os.environ["XAI_API_KEY"])
```

The text model ids on xAI's pricing page on 2026-10-12 included `grok-4.7`,
`grok-4.6`, `grok-4.5` and `grok-4.3`; that page did not say which support
tool calling, so the first live call with a tool is the test.
[^xai-models] One symptom to know: an invalid xAI key answers **400**,
"Incorrect API key provided", not 401, so a 400 on the first call is a key
problem before it is a request problem. [^raza-provider-check]

## Gemini on the free tier

A free tier is a real choice for a [demo](demo-or-production.md), with two
limits that decide the build:

- **Not every listed model is open to every account.** In the mortgage demo
  of 2026-10-12, `gemini-2.5-flash` was refused as unavailable to new users
  and `gemini-3.1-flash-lite` answered. [^raza-mortgage-demo] List the
  models with the key you will use, then make one live call to the one you
  choose, before writing its id into code.
- **The quota is small and per model.** Size the build to it: offline and
  scripted tests first, a fixed number of live calls written in
  `progress.md` before the first one, and a stop, never a silent switch to a
  paid model, when the quota runs out.

The key the owner placed on 2026-10-12 began `AQ.` rather than `AIza`, and
worked; check a Gemini key against the models endpoint rather than by its
prefix alone. [^raza-provider-check]

## Before the first call: check the key, then the endpoint

API key strings often get pasted around with the wrong label; two minutes on
the prefix saves an hour of 401s: [^af-39]

| Provider | Prefix | Example shape |
| --- | --- | --- |
| OpenAI | `sk-proj-...` or `sk-...` | 50+ alphanumeric characters after the prefix |
| DeepSeek | `sk-...` | 32 hex characters after the prefix |
| Anthropic | `sk-ant-...` | long token after the prefix |
| Google Gemini | `AIza...`, or `AQ.` on newer keys | 30-ish alphanumeric characters |

Then a read-only probe of the provider's models endpoint, which costs
nothing and says in one second whether the key and the variable name are
right; the book's example is OpenAI's, and the same idea lists the current
Gemini ids so the model string is never guessed: [^af-39]

```bash
curl -s https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY" | head -c 200
curl -s "https://generativelanguage.googleapis.com/v1beta/models" -H "x-goog-api-key: $GEMINI_API_KEY" | head -c 400
```

Send a key in a header rather than in the address where the provider
allows it; an address ends up in logs. Claude and Grok list their models
through the same OpenAI-compatible clients shown above
(`await client.models.list()`).

`load_dotenv()` must run before any module that reads these variables, or
the failure is a confusing `KeyError` deep in an import chain. [^af-39]

## Two caveats that bite

**Tracing.** A non-OpenAI model still produces traces locally, but uploading
them to OpenAI's dashboard needs an `OPENAI_API_KEY`; on a LiteLLM-only or
local setup, keep the per-run `tracing_disabled` pattern derived from whether
that key is set, or point traces at your own collector. [^af-39] The SDK
documentation lists the same choices, `set_tracing_disabled(disabled=True)`,
`set_tracing_export_api_key(...)`, or a custom processor, and names the
symptom when you forget: 401 errors from the trace upload. [^sdk-models]

**The Responses API.** Providers that do not support it need the Chat
Completions shape: `set_default_openai_api("chat_completions")`, or the
`OpenAIChatCompletionsModel` used above. Some providers also reject the
`json_schema` parameter while supporting JSON output, which can produce
malformed structured responses; test typed output against the provider you
chose before relying on it. [^sdk-models]

## Which model for which agent

Picking the right model per agent, not per app, is the biggest cost knob you
have, and the SDK makes the swap a one-line change. Default to economy and
escalate only the rows marked frontier: [^af-39]

| The work | Tier | Why |
| --- | --- | --- |
| Greetings, clarifying questions, summarising known content | Economy | No deep reasoning needed, at a fraction of the cost |
| Guardrail classifiers | Economy | "Is this a jailbreak?" doesn't need frontier power |
| High-frequency tool routing (30+ calls per conversation) | Economy | Routing is well-specified; the cheap tier handles it |
| Multi-step planning ("which 3 of 12 tools, in what order") | Frontier | Real architectural judgment pays for itself |
| Final-answer composition on high-stakes, user-facing output | Frontier | Mistakes here are visible |
| Hard reasoning: math, legal interpretation, code review | Frontier | A wrong answer is expensive to discover later |

Different agents in one app can use different models, and handoffs cross
the boundary cleanly. Where the cheap tier holds up, keep it there; where it
visibly struggles, escalate to the frontier on that specific agent. [^af-39]
Read the meter after each run with `result.context_wrapper.usage`, which
exposes `requests`, `input_tokens`, `output_tokens` and `total_tokens`.
[^af-39]

## What was verified, and what was not

Run on 2026-09-04 against openai-agents 0.22.0 with the `litellm` extra
(litellm 1.83.0), tracing disabled, one agent with one function tool and the
question "What plan is account acct_1001 on?": [^pypi-openai-agents]

| Path | Model | Result |
| --- | --- | --- |
| LiteLLM | `gemini/gemini-2.5-flash` | Reply correct; the tool was called; usage reported |
| Base-URL swap, Gemini OpenAI-compatible endpoint | `gemini-2.5-flash` | Reply correct; the tool was called; usage reported |
| Base-URL swap, local llama.cpp server (release b10797, CPU) | Qwen2.5 0.5B Instruct, Q4_K_M | Plain reply correct; the tool was not called, the model narrated it instead |

The local row is the honest limit: the wiring is proven, but a half-billion
parameter model is not a tool-calling model. For a local worker with tools,
run a model that supports function calling and re-run the probe above; the
harness does not change. [^llama-cpp] DeepSeek was not
exercised here, because this record holds no key for it; its code is the
book's, unchanged.

Run again on 2026-10-12 against openai-agents 0.23.1 (litellm 1.83.0),
tracing disabled, with the same agent and question. This record holds no
paid Claude or Grok key, so those rows prove the route, not the answer: each
was called with a deliberately invalid key, and a reply from the provider's
own authentication proves the client, the base URL and the request shape
reached the right service without spending anything. [^raza-provider-check]

| Path | Model | Result |
| --- | --- | --- |
| Base-URL swap, Gemini OpenAI-compatible endpoint, free tier | `gemini-3.1-flash-lite` | Reply correct; the tool was called; 2 requests |
| LiteLLM | `gemini/gemini-3.1-flash-lite` | Reply correct; the tool was called; 2 requests |
| Base-URL swap, Anthropic OpenAI-compatible endpoint | `claude-haiku-5-5` | 401 "Invalid Anthropic API Key" from Anthropic: route proven |
| LiteLLM | `anthropic/claude-haiku-5-5` | 401 "invalid x-api-key" from Anthropic: route proven |
| `OpenAIResponsesModel`, xAI | `grok-4.7` | 400 "Incorrect API key provided" from xAI: route proven |
| `OpenAIChatCompletionsModel`, xAI | `grok-4.7` | 400 "Incorrect API key provided" from xAI: route proven |
| LiteLLM | `xai/grok-4.7` | 400 "Incorrect API key provided" from xAI: route proven |

The first build on a real Claude or Grok key runs the probe on its own
first call and records the result in its `progress.md`; until then, a
worker on those providers has a proven route and an unproven answer, and
says so.

## What changes in the room, and what does not

| Changes | Stays |
| --- | --- |
| `.env` gains the provider's key under its own variable name | Tools, sessions, guardrails, `needs_approval`, `max_turns` |
| `models.py` exists and `build_agent()` takes `model=` | Tests, the scripted-model loop test, the golden set, the evals |
| The Vercel project variable is the provider's key (`GEMINI_API_KEY`, `ANTHROPIC_API_KEY`, `XAI_API_KEY`) instead of `OPENAI_API_KEY` | `main.py`, the Dockerfile, `Dockerfile.vercel`, `PORT=8000`, the end-to-end checks |
| A local model runs where the container runs: on your laptop or your own server, not on Vercel | Everything else on the [deploy page](fastapi-docker-vercel.md) |

Continue with the [build page](build-the-text-agent.md); every page after it
reads the same whichever model answers.

[^af-39]: Build AI Agents with the OpenAI Agents SDK: A 90-Minute Crash Course, Concept 12 "Switching models, with DeepSeek V4 Flash", "Reaching providers that aren't OpenAI-compatible: LiteLLM (any model)", the API key format check under Concept 4, and Part 6 "Cost discipline: routing by model tier". The AI Agent Factory, Panaversity, 2026.
[^sdk-models]: Models, sections on non-OpenAI models, the LiteLLM adapter, `set_default_openai_client`, tracing without an OpenAI key, and common issues (401 tracing errors, Responses API support, structured output). OpenAI Agents SDK documentation.
[^gemini-openai]: OpenAI compatibility, Gemini API documentation: base URL `https://generativelanguage.googleapis.com/v1beta/openai/`, function calling and structured output supported, "still in beta while we extend feature support". Google.
[^llama-cpp]: llama.cpp release b10797, `llama-b10797-bin-win-cpu-x64.zip`, `llama-server` serving `/v1/chat/completions`; run here with `qwen2.5-0.5b-instruct-q4_k_m.gguf` from the Qwen2.5-0.5B-Instruct-GGUF repository.
[^pypi-openai-agents]: openai-agents 0.22.0 on PyPI; the `litellm` extra installed with `uv pip install "openai-agents[litellm]==0.22.0"`, litellm 1.83.0 resolved, on 2026-09-04.
[^anthropic-openai]: OpenAI SDK compatibility, Claude API documentation, retrieved 2026-10-12: base URL `https://api.anthropic.com/v1/`; "primarily intended to test and compare model capabilities"; `strict` ignored for function calling; `response_format` ignored; `temperature` below 1 and `top_p` other than the default return 400 on Claude 4.7 and later models; `tool_choice` `"required"` or a named function returns an error on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1.
[^xai-function-calling]: Function calling, xAI documentation, retrieved 2026-10-12: `base_url="https://api.x.ai/v1"`, `os.getenv("XAI_API_KEY")`, examples on the Responses API with model `grok-4.7`, `tool_choice` `auto`, `required`, `none` or a named function.
[^xai-comparison]: Model capabilities, xAI documentation, retrieved 2026-10-12: the Responses API is the recommended way to interact with xAI models; the Chat Completions API is labelled deprecated, a legacy endpoint with limited updates.
[^xai-models]: Models and pricing, xAI documentation, retrieved 2026-10-12: text model ids including `grok-4.7`, `grok-4.6`, `grok-4.5`, `grok-4.3`; tool-calling support per model not stated on that page.
[^raza-provider-check]: Provider check, 12 October 2026: openai-agents 0.23.1 with the `litellm` extra (litellm 1.83.0), tracing disabled, one agent with one function tool, "What plan is account acct_1001 on?"; Gemini with the owner's free-tier key, which began `AQ.`; Anthropic and xAI with the string `invalid-probe-key`; results as in the table.
[^raza-mortgage-demo]: Owner's review, 12 October 2026: in the mortgage demo the coding agent reported that Google refused `gemini-2.5-flash` as unavailable to new users, and used `gemini-3.1-flash-lite`.