Tests first: red, green, refactor on one slice

The test-driven step from The AI Agent Factory: write one failing test before the code, keep the public interface tiny, test at the boundary, and let every pass be a checkpoint the agent cannot drift past.

The acceptance criteria in your spec become tests before any code exists. The book's rule for the implementation stage is strict red, green, refactor: write one failing test, watch it fail, write just enough code to make it pass, watch it pass, refactor with all tests still green, repeat. ¹

Why test-driven, specifically

Two reasons, in the book's words: ¹

It forces small steps. Without TDD, an agent produces six files of code and writes a test layer around it afterwards, and those tests tend to cheat: they exercise the implementation, not the behaviour. With TDD the test is written first, before the implementation exists, so it cannot be shaped to fit what the agent wrote.
It provides feedback every minute. Each test pass is a checkpoint. If the agent drifts, the next failing test catches it before it produces a hundred lines of garbage.

Install a test runner before you start: pip install pytest for Python. Otherwise the first red step fails on a missing runner rather than a missing implementation. ¹

The shape: a tiny interface, tests at the boundary

The book's slice shows two things visible without reading the syntax: the service has a tiny public interface, two methods, and the test calls only those two methods. It checks the behaviour a caller would see ("after three completions, the total is 30"), not how the service computes it. Small interface, wide implementation, tests at the boundary is what the book calls a deep module. ¹

The test, written first: ¹

# gamification/test_service.py - written FIRST

from datetime import datetime
from gamification.service import GamificationService, PointAward


class InMemoryStore:
    def __init__(self) -> None:
        self._events: list[PointAward] = []

    def append(self, award: PointAward) -> None:
        self._events.append(award)

    def total_for_student(self, student_id: str) -> int:
        return sum(a.points for a in self._events if a.student_id == student_id)


def test_lesson_completion_awards_ten_points():
    store = InMemoryStore()
    fixed_clock = lambda: datetime(2026, 5, 10, 12, 0, 0)
    svc = GamificationService(store, clock=fixed_clock)

    award = svc.award_lesson_completion("student-42")

    assert award.points == 10
    assert award.reason == "lesson_completion"
    assert svc.total_points("student-42") == 10


def test_multiple_completions_accumulate():
    svc = GamificationService(InMemoryStore())
    for _ in range(3):
        svc.award_lesson_completion("student-42")
    assert svc.total_points("student-42") == 30

The implementation that makes it green: ¹

# gamification/service.py - the deep module's interface

from dataclasses import dataclass
from datetime import datetime
from typing import Protocol


@dataclass(frozen=True)
class PointAward:
    student_id: str
    points: int
    reason: str
    awarded_at: datetime


class PointEventStore(Protocol):
    def append(self, award: PointAward) -> None: ...
    def total_for_student(self, student_id: str) -> int: ...


class GamificationService:
    """Awards and totals points. Streaks and levels live here too,
    but in the same module, so the interface stays small."""

    LESSON_COMPLETION_POINTS = 10

    def __init__(self, store: PointEventStore, clock=datetime.utcnow) -> None:
        self._store = store
        self._clock = clock

    def award_lesson_completion(self, student_id: str) -> PointAward:
        award = PointAward(
            student_id=student_id,
            points=self.LESSON_COMPLETION_POINTS,
            reason="lesson_completion",
            awarded_at=self._clock(),
        )
        self._store.append(award)
        return award

    def total_points(self, student_id: str) -> int:
        return self._store.total_for_student(student_id)

Two details carry the whole discipline: the store is passed in, so the test uses an in-memory one and touches no database; and the clock is passed in, so the test fixes time. Nothing in the test depends on the outside world.

What "deep" buys you when the work grows

When the next slice (a streak counter) lands, the interface gains one method and nothing else changes: the implementation grows a streak store, an activity log and a date helper, but none of that leaks out. Existing callers do nothing differently, existing tests stay green, and the new test calls only the new method. Behaviour grows; the surface barely moves. ¹

# gamification/test_service.py - existing tests untouched; ONE new test added

def test_streak_grows_with_consecutive_daily_completions():
    days = [date(2026, 5, 8), date(2026, 5, 9), date(2026, 5, 10)]
    clock = iter(datetime.combine(d, time()) for d in days)
    svc = GamificationService(InMemoryStore(), clock=lambda: next(clock))

    for _ in days:
        svc.award_lesson_completion("student-42")

    assert svc.current_streak("student-42") == 3

Apply it to your agent

Your agent's tools are the deep modules: each tool is a small interface over whatever it does, and each acceptance criterion from the spec becomes one test against that interface, written before the tool body. Give each tool its dependencies (a store, a clock, a client) as arguments so the test can hand in a fake and never touch the network.

Testing a tool

Keep the work in a plain function and make the @function_tool a thin wrapper over it, so the test calls the function directly and the SDK's decorator stays out of the unit test: ¹

# src/chat_agent/accounts.py, the deep module: a plain function...
_ACCOUNTS = {
    "acct_1001": {"name": "Ada Lovelace", "plan": "Pro", "status": "active"},
}


def find_account(account_id: str) -> str:
    account = _ACCOUNTS.get(account_id)
    if account is None:
        return f"No account found for id {account_id}."
    return f"Account {account_id}: {account['name']}, plan {account['plan']}, status {account['status']}."

# src/chat_agent/tools.py, ...and the thin wrapper the agent sees
from agents import function_tool

from .accounts import find_account


@function_tool
def lookup_account(account_id: str) -> str:
    """Look up a customer account by id and return its plan and status."""
    return find_account(account_id)

# tests/test_accounts.py, written FIRST
from chat_agent.accounts import find_account


def test_known_account_reports_plan_and_status():
    assert "plan Pro" in find_account("acct_1001")
    assert "status active" in find_account("acct_1001")


def test_unknown_account_says_so():
    assert find_account("acct_9999") == "No account found for id acct_9999."

Testing the loop without a model call

The model itself is not under test here, but the wiring around it is: does the agent call the right tool with the right argument, and does the tool's answer reach the final reply? Since release 0.21.0 the SDK ships agents.testing, deterministic test doubles for agent workflows, so that wiring runs in an ordinary test with no provider call and no key. ² A ScriptedModel plays the model: you give it the steps it will "answer" with, each step a list of output items, and the real Runner drives the real loop against it. The following was executed against openai-agents 0.22.0 on 2026-09-03: ²

# tests/test_agent_loop.py, the loop with a scripted model, no network
from agents import Agent, Runner
from agents.testing import ScriptedModel, assistant_message, function_call

from chat_agent.tools import lookup_account


def test_agent_looks_up_the_account_then_replies():
    scripted = ScriptedModel([
        [function_call("lookup_account", {"account_id": "acct_1001"}, call_id="call-1")],
        [assistant_message("Your plan is Pro and the account is active.")],
    ])
    agent = Agent(name="Test", instructions="Use the tool.", model=scripted, tools=[lookup_account])

    result = Runner.run_sync(agent, "check acct_1001")

    calls = [item for item in result.new_items if item.type == "tool_call_item"]
    assert [(c.raw_item.name, c.raw_item.arguments) for c in calls] == [
        ("lookup_account", '{"account_id":"acct_1001"}')
    ]
    outputs = [item.output for item in result.new_items if item.type == "tool_call_output_item"]
    assert outputs == ["Account acct_1001: Ada Lovelace, plan Pro, status active."]
    assert result.final_output == "Your plan is Pro and the account is active."
    scripted.assert_complete()

assert_complete() fails if a scripted step was never consumed, so a loop that stopped early is caught too. This is the tool-use record from the evals page checked by hand, and it is the test that proves the wrapper, the argument schema and the loop before any LLM-graded eval spends a model call. Run the probe on the build page first; agents.testing is one of the symbols it confirms. The finished agent's behaviour, with a real model, gets its own evals on the next pages.

Continue to build the text agent.

Agentic Engineering Fundamentals: A 45-Minute Crash Course, section 6.4 "Implementation: TDD on One Slice" and section 7 on deep modules. The AI Agent Factory, Panaversity, 2026. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸
openai-agents-python release notes, v0.21.0: "Added agents.testing ... utilities for deterministic Agent ... workflow tests". The test above was run against v0.22.0 on 2026-09-03 with no API key set. ↩ ↩²