AI Agent Development Tutorial: Tool Calling to Memory

Start with a single tool call

Most people come to AI agent development wanting to skip straight to multi-agent orchestration. The instinct makes sense — you want the model doing real work — but jumping ahead almost always means hitting a wall on state management later. Start with one tool call. Everything else builds from there.

The setup is straightforward: you declare a set of tool schemas in the system prompt, the model returns a structured tool call request in its response, your code executes that call and appends the result to the conversation, and the model generates a final answer. One synchronous request → execute → respond cycle, typically 1–3 seconds end to end.

That loop looks simple, but it establishes the fundamental contract of every agent: the model handles reasoning and decisions, the host program handles execution and environment interaction. The model never actually does anything. It expresses intent. Your code acts on it.

Here's the minimal structure using an OpenAI-compatible API:

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for up-to-date information",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-5.4",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)

tool_choice="auto" lets the model decide whether a tool call is needed. Switch it to "required" when you need guaranteed structured output — useful for data extraction pipelines where a plain-text response would break downstream processing.

---

Multi-step planning: the ReAct loop and token budget

A single tool call handles simple lookups. Real agent tasks — "research competitor pricing and write a report," for example — involve distinct phases: search, extract, summarize. That takes 4 to 12 decision steps on average.

The standard pattern for this is the ReAct loop (Reason + Act). The model outputs a Thought (its reasoning), then an Action (a tool call), receives an Observation (the result), and continues reasoning until it decides the task is done or a termination condition fires.

In code, the core mechanism is a while loop that keeps appending messages:

MAX_STEPS = 15
step = 0

while step < MAX_STEPS:
    response = client.chat.completions.create(
        model="claude-opus-4-8",
        messages=messages,
        tools=tools,
        max_tokens=4096
    )

    msg = response.choices[0].message

    # Model signals it's done
    if msg.tool_calls is None:
        break

    # Execute tools and append results
    for tool_call in msg.tool_calls:
        result = dispatch_tool(tool_call)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result
        })

    step += 1

15 steps is a reasonable ceiling in practice. Set it lower and you risk truncating complex tasks mid-execution. Set it too high and a hallucination loop will burn through tokens before you notice — Claude Opus 4.8's output token pricing is roughly 15x Haiku 4.5, so runaway loops have real cost consequences.

The other thing that bites people: context window budget. GPT-5.4 supports 128K context, but once your message list grows past ~60K tokens, the model's attention on early tool results degrades noticeably. The practical fix is summarizing older messages each loop iteration — keep the most recent N messages verbatim plus one rolling summary ahead of them.

---

Memory: three layers, three different jobs

Memory is what separates an agent that can complete a task from one that can sustain a workflow. Without it, every session starts from scratch, and multi-session knowledge accumulation is impossible.

In production systems, memory splits into three layers:

Layer	Storage	Typical Capacity	Use Case
Working Memory	messages list (in-context)	8K–32K tokens	Current task reasoning state
Episodic Memory	Vector DB (Chroma, Qdrant)	Millions of chunks	Past task results, user preferences
Semantic Memory	Structured DB or KV store	Unlimited	Domain knowledge, rules, config

Working memory lives directly in the context window — cheapest to read and write, but finite. Episodic memory uses embedding-based retrieval: chunk historical tool results into ~512-token pieces, embed them with text-embedding-3 or gemini-embedding-exp, store them in Qdrant, and run a top-5 similarity search at the start of each loop to inject relevant context into the system prompt. k=5 is a solid default starting point.

Semantic memory is closer to a traditional database. Use SQLite or Redis for structured state: task progress, confirmed user preferences, which step the agent last completed. No vector search needed — straight key-value reads.

---

State management and failure recovery

Prototype agents are fragile. Tool call timeouts, unexpected model output formats, third-party API rate limits — any one of these can drop an entire task chain. Production agents need explicit handling for all three.

At the tool execution layer, add retry logic with exponential backoff:

import tenacity

@tenacity.retry(
    stop=tenacity.stop_after_attempt(3),
    wait=tenacity.wait_exponential(multiplier=1, min=2, max=10),
    retry=tenacity.retry_if_exception_type(TimeoutError)
)
def call_tool_with_retry(tool_name: str, args: dict):
    return tool_registry[tool_name](**args)

Three attempts, exponential backoff, max 10 seconds per wait — this handles most transient network failures without over-engineering.

At the model output parsing layer, don't assume valid JSON even from frontier models. Long contexts occasionally produce malformed responses. Wrap your parser in json.loads with a try/except, and on failure, pass the raw text back as a tool result so the model can self-correct on the next step.

The most commonly skipped piece is task-level state persistence. Serialize each step as {step, tool_name, args, result, status} to a database as it executes. If the task crashes at step 22 of 30, you resume from step 22 — not step 1. On long tasks, this cuts re-run costs by 60–80%.

This architecture is exactly what tools like Claude Code, Codex, and Gemini CLI use under the hood: they maintain file system state as external memory, execute shell commands and file operations through tool calls, and inject results back into context on each iteration. Once you've built a basic loop yourself, their internals become a lot more readable — and extending them with custom tooling is straightforward.

---

When I'm testing agents across GPT-5.4, Claude Opus 4.8, and DeepSeek V3 simultaneously — whether for capability benchmarking or cost optimization — I use XycAi. One OpenAI-compatible endpoint covers 200+ global models, so Claude Code, Codex, and Gemini CLI all switch targets without a single line of code change. Flagship model pricing starts at 14% of list price, which matters a lot once your ReAct loops are running at scale.