LLM Batch API Cost Saving: OpenAI & Anthropic Guide

Why Batch Processing Cuts Your Bill in Half

Every real-time API call carries a hidden premium: you're paying for instant responses. Both OpenAI and Anthropic offer a straightforward trade-off through their batch APIs — give up strict latency guarantees, get 50% off every token. That's not a promotional rate. It's the published, official discount on both platforms, with results delivered within 24 hours.

The economics work because batch jobs fill GPU idle time. Instead of spinning up capacity on demand, providers slot your requests into off-peak windows and pass the utilization gains back to you. The math is the same on both sides: OpenAI Batch API charges 50% of standard pricing; Anthropic Message Batches do the same.

The tasks that benefit most share one trait: the result doesn't need to exist right now. Good candidates include:

• Large-scale text classification and sentiment analysis (tens of thousands of product reviews in a single run)
• Offline document summarization and pre-processing before vector indexing
• Bulk prompt evaluation for A/B tests
• Scheduled report generation (daily or weekly jobs)
• Dataset labeling and quality audits

Chatbots, real-time translation, and streaming code completion all need millisecond responses — those stay on the standard API.

---

OpenAI Batch API: From JSONL to Results

The OpenAI workflow has three steps: upload a file → create a batch → poll and download.

Step 1: Build the JSONL request file

Each line is a self-contained JSON object with a custom_id (for matching results back to inputs), method, url, and body:

{"custom_id":"req-001","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Sentiment label for: Great product, highly recommend"}],"max_tokens":20}}
{"custom_id":"req-002","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Sentiment label for: Shipping was way too slow, not happy"}],"max_tokens":20}}

Limits: 100 MB per file, 50,000 requests per batch.

Step 2: Upload and create the batch

# Upload the file
curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="batch" \
  -F file="@requests.jsonl"

# Create the batch — returns a batch_id
curl https://api.openai.com/v1/batches \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input_file_id":"file-abc123","endpoint":"/v1/chat/completions","completion_window":"24h"}'

Step 3: Poll status and download

Batches move through validating → in_progress → completed. Once you have an output_file_id, pull the results:

# Check status
curl https://api.openai.com/v1/batches/batch_xyz \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# Download results
curl https://api.openai.com/v1/files/file-output123/content \
  -H "Authorization: Bearer $OPENAI_API_KEY" -o results.jsonl

Each line in the output JSONL carries the original custom_id, so mapping results back to inputs is straightforward. Any failed requests land in a separate error_file_id — always check it.

---

Anthropic Message Batches: SDK-First

Anthropic's batch API is designed around its Python SDK. You can use REST, but the SDK path is cleaner.

Submit a batch

import anthropic

client = anthropic.Anthropic()

batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": "review-001",
            "params": {
                "model": "claude-haiku-4-5",  # lightweight model, lower cost per batch
                "max_tokens": 50,
                "messages": [{"role": "user", "content": "Sentiment analysis: Fast shipping, arrived intact"}]
            }
        },
        {
            "custom_id": "review-002",
            "params": {
                "model": "claude-haiku-4-5",
                "max_tokens": 50,
                "messages": [{"role": "user", "content": "Sentiment analysis: Item arrived damaged, poor support"}]
            }
        }
    ]
)
print(batch.id)  # msgbatch_xxxxxxxx

Poll and process results

import time

while True:
    status = client.messages.batches.retrieve(batch.id)
    if status.processing_status == "ended":
        break
    time.sleep(60)  # once per minute is plenty

for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        print(result.custom_id, result.result.message.content[0].text)
    else:
        print(f"Failed: {result.custom_id} — {result.result.error}")

Anthropic's limits are tighter: 10,000 requests or 32 MB per batch. For very large jobs you'll need to shard the input yourself before submitting.

---

Side-by-Side Comparison

	OpenAI Batch API	Anthropic Message Batches
Discount	50%	50%
Max requests per batch	50,000	10,000
Max file size	100 MB	32 MB
Result window	24 hours	24 hours
Primary interface	REST + JSONL file	SDK-first / REST
Supported models	GPT-4.5, GPT-4.1, GPT-4.1-mini, etc.	Claude Opus 4, Sonnet 4.5, Haiku 4.5
Error handling	Separate error file	type field in result stream
Cancellation	Supported	Supported

How to choose: If your jobs regularly exceed 10,000 requests per batch or you already have a JSONL pipeline, OpenAI's higher limits make it the natural fit. If your team runs Python and your batch sizes are moderate, Anthropic's SDK interface is more concise and its inline error handling is easier to reason about. In practice, there's no reason to pick just one — run cost-sensitive classification on GPT-4.1-mini or Haiku 4.5, and reserve GPT-4.5 or Opus 4 for tasks that need heavier reasoning.

---

Three Things to Do Before Going to Production

1. Design for idempotency. Use globally unique custom_id values — a format like {task_type}-{record_id}-{timestamp} works well. This lets you safely resubmit failed batches without double-processing records that already succeeded.

2. Monitor your failure rate. A healthy batch should fail under 1% of requests. If a batch hits 5% or higher, check whether your prompts are tripping content policies, then verify that your max_tokens setting isn't so low it's truncating valid responses.

3. Estimate costs before you commit. Run 100 requests through the standard API first, measure average token consumption, then multiply by the batch price and your full volume. Knowing that 50,000 requests × 500 average tokens × GPT-4.1-mini batch pricing comes out to $X before you submit is a lot better than reading the invoice afterward.

---

The batch discount logic is simple. Where teams usually get stuck is the operational layer — managing multiple API keys, switching between providers, and keeping billing visible across models. My team routes everything through XycAi, a single OpenAI-compatible endpoint covering 200+ models from OpenAI, Anthropic, DeepSeek, and others. GPT and Claude official models start from 14% of list price, which makes a real difference on high-volume batch workloads — worth a look if you're running this kind of pipeline at scale.