FastAPI, Docker, Vercel: put the agent behind a URL

"Works on your laptop" means something specific: the agent runs as a Python process you started by hand, reads its key from a file in the project folder, writes its state to a local file, and only you can reach it. Production breaks every one of those assumptions. Real users reach the agent over the public internet, many at once; its state has to survive the host restarting; its secrets have to stay out of reach of anything it generates; and each run has to be observable. Production deployment is not a thin wrapper around "works on my laptop"; it is a different architecture. ¹

The book's full course builds that architecture with a database, file storage and a code sandbox. The basic worker in this room takes the first three steps of it, in the book's own order: scaffold the FastAPI harness, containerize it, deploy it. The book designs the harness to boot with only OPENAI_API_KEY set; every other backend is optional and off until its key exists, and the agent still runs. ² The deployment target here is Vercel, from Vercel's own documentation, because the book's target needs a cloud account this room does not assume.

The one rule to carry

The harness is the agent's brain: it receives requests, runs the agent loop, holds the secrets and keeps the record. A sandbox is the agent's hands, where generated code runs, with no access to the harness's secrets. The original sin of AI deployment is running agent-generated code inside the harness, next to the model key and your users' data. ¹ The text agent in this room generates no code and runs none: its tools are plain Python functions whose bodies run in the harness process, which the book's companion allows on purpose. ² If a later version of your worker needs to run code, that is the point to read the book's sandbox chapters, not to run it in the harness.

FastAPI: the door

FastAPI turns your Python code into something the network can talk to. The vocabulary: an endpoint is a path your API handles, like POST /runs to start a task or GET /health to check the harness is alive; a route handler is the function that runs when it is called, marked with a decorator such as @app.post("/runs"); async def and await let one process handle hundreds of waiting requests at once, and most of the harness's work is waiting on the model; Pydantic models describe the shape of request and response data, so malformed requests are rejected before your code runs; Uvicorn is the program that runs the app and connects the network to your handlers. ¹

The book chooses FastAPI for three reasons that fit this SDK: the SDK is built on asyncio, so handlers await it directly; FastAPI generates an OpenAPI schema from the type hints, so the eval suite can hit the endpoints with checked requests and the API documents itself; and it validates with Pydantic, the same library the SDK uses internally. What it is not: a general web framework, and not a queue; a task longer than a request can stay open gets queued and the client checks back. ¹

The harness, two files

The book's companion main.py has two endpoints: GET /health, which reports which backends are active, and POST /runs, which loads the session, runs the agent, persists the run and returns the reply. It imports even without a key, so the harness boots before any key is added, and /runs answers 503 with a clear message until OPENAI_API_KEY exists. ² Below is that file with its database, storage and sandbox layers removed, because this room's agent has none; the session comes from the SDK's own SQLiteSession, as on the build page.

First move the Agent(...) construction out of cli_v2.py into a factory, the companion's shape, so the CLI and the harness share one agent: ²

# src/chat_agent/agent.py, one factory the CLI and the harness both call
from agents import Agent

from .tools import book_meeting  # the typed tool from the build page


def build_agent() -> Agent:
    """Construct the agent. No model named: the installed SDK's default is used."""
    return Agent(
        name="Chatty",
        instructions="You are a friendly conversational assistant. Be concise.",
        tools=[book_meeting],
    )

Add the guardrails and the approval-gated tool from the build page to the same constructor. Then the harness:

# src/chat_agent/main.py, the harness. Boot with: uvicorn chat_agent.main:app
#   GET  /health -> liveness
#   POST /runs   -> run the agent for one message and return the reply
from __future__ import annotations

import os

from dotenv import load_dotenv

load_dotenv()  # before any module that reads environment variables

from agents import Agent, Runner, SQLiteSession
from agents.models import get_default_model
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

from .agent import build_agent

agent: Agent = build_agent()
app = FastAPI(title="Chat agent harness")


class RunRequest(BaseModel):
    message: str
    session_id: str = "default"


class RunResponse(BaseModel):
    reply: str


@app.get("/health")
async def health() -> dict:
    return {"status": "ok", "model": get_default_model()}


@app.post("/runs", response_model=RunResponse)
async def create_run(req: RunRequest) -> RunResponse:
    if not os.getenv("OPENAI_API_KEY"):
        raise HTTPException(
            status_code=503,
            detail="OPENAI_API_KEY is not set. Add it to .env to run the agent.",
        )
    session = SQLiteSession(req.session_id, "conversations.db")
    result = await Runner.run(agent, req.message, session=session, max_turns=10)
    return RunResponse(reply=str(result.final_output))

/health reports the model the installed SDK will use, read from the SDK itself rather than typed into the page, because the default changes between SDK releases; the build page says which version this room was checked against and how to re-check. Install the two dependencies, then boot: ²

uv add fastapi "uvicorn[standard]"
uv run uvicorn chat_agent.main:app --reload --port 8000

uv add records the version it installed as the floor in pyproject.toml; the companion's own floors are fastapi>=0.115 and uvicorn[standard]>=0.34, both satisfied by current releases. ²

The book's done-when for this step: the harness starts with no errors, GET /health returns {"status": "ok", ...} on a bare key-only boot, and GET /docs shows the auto-generated API for the two endpoints. ¹ That boot is the early win the book promises: before any cloud account, any Docker, any database, a real agent harness answering from your own laptop. ¹

Docker: the sealed bundle

A container is a sealed bundle of your app plus everything it needs to run. Build it once, run it anywhere a container engine runs, and "it works on my machine" stops being a complaint. A Dockerfile is the recipe; a base image is the starting point; a multi-stage build installs dependencies in a cached layer before the source is copied, so the image stays small and a source change rebuilds fast. ¹

The book's Dockerfile, from its companion, with only the module name changed to this room's package: ²

# Dockerfile. Slim Python base, uv for install.
FROM python:3.12-slim

# uv: fast, reproducible installs from the lockfile.
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /app

# Install deps first (cached layer) before copying the source.
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-install-project

# Now copy the source and install the project itself.
COPY src ./src
RUN uv sync --frozen --no-dev

ENV PATH="/app/.venv/bin:$PATH"

EXPOSE 8000

# The platform terminates TLS at its ingress, so the app
# serves plain HTTP and trusts the forwarded headers.
CMD ["uvicorn", "chat_agent.main:app", "--host", "0.0.0.0", "--port", "8000", "--proxy-headers"]

The book's instruction for the same step adds a .dockerignore that excludes the virtualenv, caches and .env files, and says to build the image and run it locally with your .env mounted: ¹

# .dockerignore
.venv
__pycache__
.env

docker build -t chat-agent .
docker run --rm -p 8000:8000 --env-file .env chat-agent

Done when: the image builds with no errors, the container runs locally and GET /health returns ok from inside it, and changing a source file and rebuilding is fast because the dependency layer stays cached. ¹

Vercel: from the official docs

Vercel Functions can run OCI-compatible container images. Get started by creating a Dockerfile.vercel (or Containerfile.vercel) file placed at the root of your project; Vercel automatically detects these files and adds a rewrite rule to route all traffic to the container image. ³ So the first step is a copy of the book's Dockerfile under that name:

cp Dockerfile Dockerfile.vercel

Port. Functions running container images are expected to open an HTTP server. The default port is 80, and it can be overridden by setting the PORT environment variable in the project settings. ³ The book's Dockerfile listens on 8000, so set PORT to 8000 on the project rather than editing the file.

Environment variables. Environment variables are key-value pairs configured outside your source code; they are encrypted at rest, and it is safe to store sensitive data such as tokens in them. For each variable you choose the environments it applies to: Production (your next production deployment, created by pushing to the production branch or running vercel --prod), Preview (created by pushing to another branch or running vercel), and Development (used by vercel dev). Any change to environment variables is not applied to previous deployments; it only applies to new deployments. ⁴ Add two variables to the project:

Deploy. Deploy your application using the Vercel CLI or by pushing to a Git repository. During the build step, the image will be built and pushed to Vercel Container Registry. ³ With the CLI, vercel alone is the deploy command; when deploying, standard output is always the deployment URL. The first deployment of a new project is always a production deployment, even when you omit --prod; use --prod for later production deployments after that first one exists. ⁵

vercel                # first deploy: links the project, builds the image, prints the URL
vercel --prod         # later production deploys
vercel > deployment-url.txt   # stdout is always the deployment URL

What the person is handed. The build is not finished when the deploy command exits; it is finished when the person can open it. Hand over three lines, in plain words, at the end of this step and again in the final report: the live address to open or share (the deployment URL, with /health checked from another device), the folder on disk where the code lives (the absolute path of the project), and the one command that runs it again locally. A person who cannot find the code or the address has not received a worker. The address opens the worker's page, not an API documentation page: a password, an upload, a Run button and a chat box, so the hand-over also says where the password lives, never the password itself. OWNER_PASSWORD, SESSION_SECRET and DAILY_CHAT_LIMIT join the model key in the project's environment variables.

Local run of the container. You can also develop and test locally using vercel dev; this requires the docker CLI and Docker daemon to be available on your machine. ³

What happens when nobody calls it. Functions not receiving any traffic for 5 minutes in production environments, and 30 seconds in preview environments, automatically scale down. When an instance is scaled down, the container receives a SIGTERM signal with a 30 seconds grace period to clean up before being forcefully terminated. ³ Runtime logs from the container's stdout and stderr are available in the dashboard; the same limits that apply to Vercel Functions apply to custom container images, and Secure Compute and Static IPs are not yet supported with them. ³

When a frontend joins later. Services let you deploy multiple backends and frontends within a single Vercel project, for example a Next.js frontend and a FastAPI backend that deploy together with shared routing and environment variables. A service is internal by default and is not routable from the internet unless a top-level rewrite targets it as destination. ⁶ For a container service, set the service's entrypoint to the path of its Dockerfile relative to its root: ³

{
  "services": {
    "frontend": {
      "root": "frontend/",
      "entrypoint": "Dockerfile.vercel"
    },
    "backend": {
      "root": "backend/",
      "entrypoint": "Dockerfile.vercel"
    }
  },
  "rewrites": [
    { "source": "/api/(.*)", "destination": { "service": "backend" } },
    { "source": "/(.*)", "destination": { "service": "frontend" } }
  ]
}

Run all services together locally with vercel dev, or vercel dev -L to run without authenticating with the Vercel cloud. ⁶ The single Dockerfile.vercel at the root is enough for this room; reach for services only when a second application shares the project.

Continue to test end to end.