How to Control LLM Output Length and Cut Token Costs

After running LLM APIs in production for a few months, I pulled the billing dashboard and found that roughly 70% of tokens were going toward the model explaining itself — restating questions, adding context nobody asked for, wrapping up with a polite summary. The same question that got an 800-token response from GPT became a 200-token response after some prompt adjustments, with no meaningful loss of information.

Controlling LLM output length isn't an advanced technique. It's just something most teams never sit down and do systematically. Here are five methods that actually work, with tradeoffs for each.

1. max_tokens Is a Hard Ceiling — Use It With Format Instructions

Most people set max_tokens and assume the problem is solved. It isn't. The parameter truncates output; it doesn't refine it. The model has no idea you're cutting it off at token 300, so it might write the key conclusion at token 350 and you'll never see it.

The fix is pairing max_tokens with explicit format instructions that tell the model what shape the answer should take:

response = client.chat.completions.create(
    model="gpt-5.4-mini",
    max_tokens=300,
    messages=[
        {
            "role": "system",
            "content": "You are a concise technical assistant. Answer in 3 sentences max. Lead with the conclusion. Skip background context."
        },
        {"role": "user", "content": user_input}
    ]
)

For the Anthropic API, max_tokens is a required field. Set it by task type rather than picking one generous default: ~150 for summaries, ~250 for Q&A, ~600 for code generation.

Real-world comparison: "Explain Redis cache penetration" with no constraints got ~520 tokens from Claude Sonnet. Adding max_tokens=250 plus format instructions brought it to 210 tokens with all the core information intact.

2. Bake Brevity Into the System Prompt

The system prompt is the most stable lever for controlling output length. Instead of asking for a short answer on every user turn, make conciseness part of the model's default personality from the start.

Vague instructions like "be concise" don't work well. Specific behavioral constraints do:

You are an API documentation assistant.
Rules:
- Answers contain only conclusions and necessary code. No "This is because..." explanatory paragraphs.
- Bullet lists cap at 5 items, each under 15 words.
- Never restate the user's question at the start of your answer.
- Never add a summary sentence at the end.
- If the question can be answered in one sentence, use one sentence.

The "never restate the question" rule is worth highlighting on its own. Models have a consistent habit of opening with "You're asking about..." before they get to the actual answer. That's pure token waste. On a high-traffic endpoint, it adds up to thousands of extra tokens per day.

Gemini Flash in particular tends toward verbose output. Adding Never restate the user's question to the system instruction makes an immediate, measurable difference.

3. Use Structured Output to Eliminate Prose Overhead

Returning JSON instead of natural language is one of the most aggressive compression techniques available. Prose burns tokens on transition phrases, hedging language, and politeness. JSON has none of that.

OpenAI's API supports response_format: { type: "json_object" } and full Structured Outputs via JSON Schema. Here's what the difference looks like for a sentiment analysis task:

Natural language (~85 tokens):

The text expresses a generally positive sentiment. The user appears satisfied with the product's functionality but shows mild negative sentiment toward the pricing. Overall tone is positive.

JSON (~28 tokens):

{"sentiment": "positive", "score": 0.72, "price_concern": true}

Same information, 33% of the token cost. For batch processing jobs, that gap shows up directly on your invoice.

This also applies to coding tools like Claude Code and Codex — ask for diff output or a bare code block, not a narrated walkthrough of the changes.

4. Anchor Output Length With Few-Shot Examples

"Keep it short" is a vague instruction. A short example is a precise one. The length of your few-shot examples sets the model's expectation for how long its answers should be. Give 50-word examples and you'll mostly get 50-word answers.

Example Q: How do you reverse a string in Python?
Example A: s[::-1]

Example Q: What is idempotency?
Example A: An operation that produces the same result whether run once or many times. HTTP GET and PUT are idempotent; POST is not.

Now answer: {user_question}

Those two examples cost under 60 input tokens and reliably anchor the model's output volume. DeepSeek V3 and Qwen 3 respond especially well to this — both have high format-following accuracy and won't suddenly produce a 600-word essay after being shown two-sentence examples.

One caveat: few-shot examples increase input token cost, so this is only worth doing for high-frequency, fixed-format endpoints. Don't add it to one-off queries.

5. Streaming With Early Termination

The first four methods work on the input side. This one works on the output side. For latency-sensitive applications, streaming with early cutoff gives you another dimension of control.

import re

stream = client.chat.completions.create(
    model="claude-opus-4-8",
    stream=True,
    messages=[...]
)

collected = []
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    collected.append(delta)
    full_text = "".join(collected)
    # Stop receiving once we detect a semantic endpoint
    if re.search(r'[.!?]\n|(\n\n)', full_text) and len(full_text) > 100:
        stream.close()
        break

The logic: once you detect a natural stopping point — a double newline, a sentence-ending punctuation followed by a line break — close the stream before the model appends its summary or follow-up remarks. For Q&A endpoints, this typically cuts output tokens by 15–25%.

---

How Much Each Method Actually Saves

Method	Best For	Typical Reduction	Implementation Effort
max_tokens + format instructions	All endpoints	20–40%	Low
System prompt brevity persona	Fixed-function interfaces	25–50%	Low
Structured JSON output	Extraction, classification	40–70%	Medium
Few-shot length anchoring	High-frequency, fixed format	20–45%	Medium
Streaming early termination	Real-time chat, Q&A	15–25%	High

These methods compound. A production customer support endpoint running all three of the low-to-medium effort techniques — brevity persona, JSON output, and max_tokens=200 — dropped monthly output tokens from 120 million to 48 million. A 60% reduction in the LLM line item, with no increase in support complaints about answer quality.

Start with the system prompt and max_tokens since they cost nothing to implement. Add structured output wherever you're doing extraction or classification. Layer in few-shot anchoring for your highest-traffic endpoints. Streaming termination is worth the engineering effort if you're running a real-time product and every millisecond counts.

---

If you're A/B testing these parameters across multiple models, I'd recommend trying XycAi. It's a single OpenAI-compatible API that covers 200+ models — GPT, Claude Opus 4.8, Gemini 3, DeepSeek V3 — with official model pricing starting at 14% of list price. You can tune max_tokens and compare outputs across model families without juggling separate API keys, and all your token usage rolls up into one dashboard.