Multi-Model Prompt Comparison: GPT, Claude, Gemini & More

Why the same prompt produces wildly different results

You've probably seen this: send the same request to three different models and get back a 2,000-word essay from one, three lines of working code from another, and a confident answer to a question you didn't actually ask from the third.

That's not random noise. The differences are structural. Each model carries the fingerprint of its pretraining data distribution, its RLHF reward signals, how it utilizes its context window, and how finely it parses instructions. More fundamentally, vendors don't agree on what a good response looks like. GPT-5.5 leans toward complete, hard-to-misread explanations. Claude Opus 4.8 tends to surface ambiguity before answering. Gemini 3 Pro gravitates toward structured output. DeepSeek R1 makes its reasoning chain visible by default.

If you're routing different tasks to different models — or trying to decide which one to standardize on — you need a comparison grounded in actual outputs, not marketing benchmarks. This article walks through a fixed prompt set run across six mainstream models, with a scoring framework you can reuse and concrete selection recommendations at the end.

---

The testing setup: 3 prompt types, 6 models

Models tested

Vendor	Model	Profile
OpenAI	GPT-5.5	Flagship general
Anthropic	Claude Opus 4.8	Flagship general
Google	Gemini 3 Pro	Balanced general
DeepSeek	DeepSeek R1	Reasoning / coding
Zhipu	GLM-5.2	Chinese-first general
Alibaba	Qwen 3 72B	Bilingual general

How the prompts were chosen

Random sampling is useless here. You want prompts that stress-test three distinct capabilities:

A. Open-ended generation — tests style control and information density

Explain the transformer attention mechanism in 200 words for someone who codes but has never studied deep learning.

B. Structured reasoning — tests logical completeness and step integrity

Write a Python function that takes a list and returns all pairs whose product exceeds 100. Include a time complexity analysis.

C. Edge case handling — tests refusal behavior and ambiguity resolution

Write a resignation letter. Tone: blunt, no softening.

Each prompt ran 3 times per model. I used the second run (the first can be affected by caching; the third sometimes drifts). Temperature was fixed at 0.7 across all models, no max_tokens cap.

---

What the outputs actually looked like

Prompt A: Explain attention in 200 words

Word counts: GPT-5.5 averaged 310 words and flagged that it had exceeded the limit. Claude Opus 4.8 came in at 198 — essentially on target. Gemini 3 Pro averaged 260. DeepSeek R1 averaged 420, partly because it included a visible reasoning trace. GLM-5.2 averaged 185. Qwen 3 72B averaged 220.

The meaningful differences were qualitative:

• GPT-5.5 opened with an anchor analogy (attention as a search engine relevance score) before expanding. High readability, weak length discipline.
• Claude Opus 4.8 caught the "someone who codes" constraint and used pseudocode to bridge the gap — without going over the word limit. Most consistent instruction-following of the group.
• DeepSeek R1's explanation was genuinely good, but roughly 140 words were meta-commentary on its own reasoning process. Useful for debugging the model; noise for end users.
• GLM-5.2 produced the most natural prose in Chinese contexts, but the technical depth was lower than warranted for a reader who "codes." Better suited to a true beginner than the stated audience.

Prompt B: Python function + complexity analysis

The real test here is whether the model defaults to the brute-force O(n²) solution or proactively offers something better.

Model	Approach	Complexity analysis	Runs clean?
GPT-5.5	Brute force + itertools version	Correct, distinguishes best/worst case	✅
Claude Opus 4.8	Optimal only, explained why it skipped brute force	Correct	✅
Gemini 3 Pro	Three versions: brute / optimized / numpy	Correct	✅
DeepSeek R1	Brute force + mathematical proof	Correct, most detailed	✅
GLM-5.2	Brute force only	Partially correct — missed space complexity	✅
Qwen 3 72B	Brute force + pruned version	Correct	✅

All six produced runnable code. By 2025–2026 standards, baseline code correctness is table stakes for major models. The differentiation is in whether the model proactively moves toward a better solution and how thoroughly it explains the tradeoffs.

Prompt C: Blunt resignation letter

This prompt has deliberate emotional tension. I wanted to see how each model handles "don't soften this."

GPT-5.5 and Gemini 3 Pro both delivered the letter, then appended a disclaimer along the lines of "you may want to reconsider the tone before sending." Claude Opus 4.8 asked one clarifying question — "what outcome are you hoping this letter achieves?" — then wrote two versions based on an assumed answer. DeepSeek R1 and Qwen 3 72B completed the task without comment. GLM-5.2's output was the most direct, but some phrasing crossed into territory that would read as professionally damaging in most workplace contexts.

None of these is wrong, exactly. But they reflect real differences in how each model weights user autonomy against perceived risk.

---

Scoring framework and model selection

Four dimensions worth measuring explicitly:

Instruction Following Rate (IFR): How reliably does the model honor explicit constraints — word count, format, audience definition? Claude Opus 4.8 hit roughly 91% across tests. GPT-5.5 came in around 78%, mostly because it expands beyond stated limits when it judges that more context helps.

Output density: Useful information per token. DeepSeek R1 had the highest raw token count, but strip out the reasoning trace and its effective density drops to mid-range. Claude Opus 4.8 and Qwen 3 72B were the most efficient.

Edge case handling: Claude Opus 4.8 clarifies before answering. GPT-5.5 answers then annotates. DeepSeek and Qwen execute without commentary. Which behavior you want depends entirely on the use case.

Chinese language quality: GLM-5.2 and Qwen 3 72B are noticeably better for native Chinese output — long-form content, register control, idiomatic phrasing. The gap versus Western-origin models is real and consistent.

Selection guide:

• Strict format constraints or structured workflows → Claude Opus 4.8
• Code generation with visible reasoning → DeepSeek R1, or GPT-5.5 via Codex CLI
• Long-context multilingual document work → Gemini 3 Pro
• Chinese content production → Qwen 3 72B or GLM-5.2
• Tasks where you want the model to proactively explore alternatives → GPT-5.5

---

A reusable testing methodology

The most common mistake in multi-model prompt comparisons is inconsistent conditions — different temperature settings, tests run days apart when a model may have been silently updated. Your results become incomparable.

Standard procedure that holds up:

1. Fix temperature at 0.7 (stable enough for repeatability, loose enough that differences surface)
2. Run each prompt at least 3 times; use the middle result
3. Log input + output token counts for cost modeling
4. Use an OpenAI-compatible API across all models to eliminate SDK-level noise
5. Score against a rubric — at minimum: accuracy, format compliance, information density, edge case handling

Here's a minimal Python harness that works across any OpenAI-compatible endpoint:

import openai
import time

def test_prompt(client, model, prompt, runs=3):
    results = []
    for i in range(runs):
        resp = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            temperature=0.7
        )
        results.append({
            "run": i + 1,
            "content": resp.choices[0].message.content,
            "tokens": resp.usage.total_tokens
        })
        time.sleep(1)  # stay inside rate limits
    return results[1]  # second run

Because this uses the OpenAI-compatible interface, you can swap the model name and base URL without touching anything else. That's what makes systematic multi-model comparison practical rather than tedious.

---

After running this kind of comparison a few times, the most efficient setup I've found is a single API entry point that routes to whichever model fits the task. That's exactly what XycAi provides — one OpenAI-compatible interface covering 200+ models globally, with GPT-5.5 and Claude Opus 4.8 available from 14% of list price. Claude Code, Codex CLI, and Gemini CLI all connect with a single command, which makes switching between models for testing — or production routing — significantly less painful than managing separate integrations for each vendor.