LLM Function Calling Tutorial: A Production-Ready Guide

Most LLM function calling tutorials stop at the demo stage — they hand the model a get_weather function, show it returning JSON, and call it a day. Real production environments are a different animal: auth failures, retry logic, concurrent tool calls, output validation, and the perennial headache of a model hallucinating a parameter value that doesn't exist anywhere in your system. This guide skips the toy examples and works through the full chain from schema design to result validation.

Schema Design: Where Model Behavior Starts

A poorly written schema means every downstream fix is just patching symptoms. A few high-leverage principles:

Treat the description field as a prompt, not a comment. Description quality has a direct, measurable effect on call accuracy. Compare these two:

// ❌ Useless
"description": "Query an order"

// ✅ Useful
"description": "Fetch status and shipping info for a given order ID. Only valid for created orders — not draft orders. Order IDs follow the format 'ORD-' followed by exactly 8 digits."

The second version encodes when to call the function, what input format to expect, and where the edge cases are. In internal A/B testing, this kind of description brought the incorrect-call rate down from roughly 18% to under 3%.

Use enum to lock down discrete values. If a parameter has a fixed set of valid options, don't use string and explain the options in prose — just use enum:

"status": {
  "type": "string",
  "enum": ["pending", "processing", "shipped", "delivered", "cancelled"],
  "description": "Current order status"
}

Be deliberate about required vs. optional. Only put fields in the required array that are genuinely non-negotiable. Give optional parameters default values and document the fallback behavior in their descriptions. Every unnecessary required field is another chance for the model to ask a useless clarifying question or make a bad call when that information isn't available.

Cap nesting at two levels. Beyond two levels of schema nesting, JSON generation accuracy drops noticeably. Flatten deep structures into prefixed field names instead.

Error Handling: try-catch Isn't Enough

Function calling errors fall into three categories, each requiring a different response:

Error type	Example	Strategy
Model format error	Wrong param type, missing required field	Validate and return a structured description so the model can self-correct
Business logic error	Order not found, insufficient permissions	Return a structured error code with an actionable suggestion
Infrastructure error	Timeout, 503	Retry with exponential backoff; abort and report if threshold is exceeded

The first category is where most teams get it wrong: don't throw a raw exception stack at the model. The model can't parse a Java stacktrace, but it can absolutely work with a structured error response. This is the pattern that works:

{
  "success": false,
  "error_code": "ORDER_NOT_FOUND",
  "error_message": "Order ORD-12345678 does not exist. Check that the order ID is correct.",
  "suggestion": "Use the search_orders tool to look up orders by user ID instead."
}

The suggestion field is easy to overlook but it matters: it gives the model a next move, keeping the conversation going instead of hitting a dead end. In practice, this pattern improves multi-turn tool call completion rates by around 25%.

For retries, exponential backoff is the right default: start at 500ms, double each time, cap at 3 retries, and enforce a 10-second total timeout. If you exceed that, fail gracefully and tell the user. Infinite retries will hang the entire conversation.

Concurrent Calls: Actually Using Multi-Tool Responses

Both GPT and Claude Sonnet support returning multiple tool calls in a single response. If your code is still processing them serially, you're leaving significant performance on the table.

The tool_calls array can contain several independent calls — for example, fetching order status and user profile simultaneously. Run them concurrently:

import asyncio

async def execute_tool_calls(tool_calls: list) -> list:
    tasks = []
    for call in tool_calls:
        task = asyncio.create_task(
            dispatch_tool(call.function.name, call.function.arguments)
        )
        tasks.append((call.tool_call_id, task))

    results = []
    for tool_call_id, task in tasks:
        result = await task
        results.append({
            "role": "tool",
            "tool_call_id": tool_call_id,
            "content": json.dumps(result)
        })
    return results

Check dependencies before parallelizing. get_user_info and get_order_list can run concurrently. But if the second call needs a value from the first, they have to be sequential. You can steer the model toward the right ordering by adding guidance in the schema description: "Requires a user_id — call get_user_info first." This prompts the model to split them across turns rather than firing them simultaneously.

In I/O-bound workloads, concurrent tool execution typically cuts total response time by 60–75%. It's one of the highest-ROI optimizations in a production function calling setup.

Output Validation: Don't Trust Model-Generated Parameters Blindly

Model-generated function call arguments need an independent validation pass before execution. The reason is straightforward: models hallucinate. Common patterns include fabricating a plausible-looking but nonexistent order ID, passing a numeric string where an integer is required, or inventing an enum value that sounds reasonable but isn't in the schema.

JSON Schema validation with jsonschema is the lightest-weight solution in Python:

from jsonschema import validate, ValidationError

def safe_dispatch(function_name: str, raw_arguments: str) -> dict:
    try:
        args = json.loads(raw_arguments)
        schema = TOOL_SCHEMAS[function_name]["parameters"]
        validate(instance=args, schema=schema)
    except json.JSONDecodeError:
        return {"success": False, "error_code": "INVALID_JSON"}
    except ValidationError as e:
        return {
            "success": False,
            "error_code": "SCHEMA_VALIDATION_FAILED",
            "error_message": e.message
        }

    return execute_function(function_name, args)

Validate outputs too, especially when a tool result feeds into another tool. Define a ResponseSchema and treat it with the same rigor as input schemas. Don't assume your internal APIs always return well-formed data.

One more thing that often gets skipped: idempotency. Any write operation — placing an order, processing a payment, sending a message — needs to support an idempotency key. Models will retry calls when they're uncertain, and without idempotency protection you'll end up with duplicate orders. The fix is simple: generate a UUID per tool call and pass it to the business API as the idempotency key.

Putting the Whole Chain Together

Production-grade function calling isn't about any single technique — it's about getting schema quality, error structure, concurrent execution, and input validation right at the same time. Any one of those gaps will surface under load or edge-case inputs.

Start small: wire up two or three tools, run them through at least 500 real conversations, then expand. Once your tool count exceeds ten, add a routing layer — have the model first select a relevant subset of tools from the full registry, then execute. Without this, schema context starts eating a large chunk of your token budget and selection accuracy suffers.

---

When I'm testing function calling behavior across Claude Sonnet, GPT, and DeepSeek, I run everything through XycAi. One OpenAI-compatible endpoint covers 200+ models, so switching between them for comparison testing requires zero code changes. For a workflow that involves repeated iteration across multiple models, the pricing — official Claude and GPT models at 14% of list price — makes a real difference in what you can afford to test.