AI Document Summarization for Long PDFs: Map-Reduce Pipeline

The Problem: Long PDFs Hit a Hard Ceiling

A 200-page legal contract. A 500-page technical white paper. Drop either one directly into a model and you'll hit a wall fast.

Context windows are larger than ever — Claude Opus 4.8 handles 200K tokens, GPT-5.4 tops out at 128K, and Gemini 3 pushes to 1M. That sounds generous until you factor in what actually happens to real documents: OCR artifacts from scanned pages, double-column layouts, appendices stuffed with tables. A 100-page PDF can easily clear 80K tokens before you've processed anything interesting.

Then there's cost. Feeding a 300-page report into Opus 4.8 in a single request can run over $2 in input tokens alone. Do that at scale and the economics collapse immediately.

The sustainable engineering answer is chunking plus aggregation — specifically, a map-reduce summarization pipeline. Slice the PDF into chunks, summarize each one independently (map), then synthesize all the chunk summaries into a final output (reduce). This piece walks through a production-ready implementation in Python and LangChain that works with any OpenAI-compatible API.

---

Chunking Strategy: Cut at Semantic Boundaries, Not Arbitrary Lengths

Chunk quality is what determines summary quality. The most common mistake is splitting on fixed character counts, which tears paragraphs apart and bisects tables mid-row.

Here's a practical hierarchy of chunking strategies:

Strategy	Best for	Recommended size
Heading / section boundaries	Reports and papers with clear structure	Natural section size, no forced limit
Recursive paragraph splitting	General-purpose PDFs with unknown structure	1,500–2,500 tokens, 200-token overlap
Semantic similarity clustering	Documents that jump between topics frequently	Dynamic size, higher compute cost
Fixed token splitting	Fallback only	~2,000 tokens, not a first choice

Overlap is the parameter most people underestimate. Setting a 200-token overlap between consecutive chunks preserves cross-boundary context — causal relationships, pronoun references, conditional clauses that span a paragraph break. In testing on legal documents, increasing overlap from 0 to 150 tokens improved logical coherence scores (via human evaluation) by roughly 18%.

Here's the core extraction and chunking code using PyMuPDF and LangChain:

import fitz  # PyMuPDF
from langchain.text_splitter import RecursiveCharacterTextSplitter

def extract_and_chunk(pdf_path: str, chunk_size: int = 2000, overlap: int = 200):
    doc = fitz.open(pdf_path)
    full_text = ""
    for page in doc:
        full_text += page.get_text("text")

    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=overlap,
        separators=["\n\n", "\n", ".", " "],
        length_function=len,  # swap in tiktoken for accurate token counts
    )
    chunks = splitter.split_text(full_text)
    print(f"Document split into {len(chunks)} chunks")
    return chunks

---

Building the Map-Reduce Pipeline

Map phase: Hit each chunk concurrently with a lightweight model and get back a local summary. This is where you save money. GPT-5.4-mini or Claude Haiku 4.5 cost less than a tenth of their flagship counterparts per million tokens, and chunk-level summarization doesn't need flagship-level reasoning.

Reduce phase: Concatenate all chunk summaries and pass them to a flagship model — Opus 4.8 or GPT-5.5. This is where global synthesis matters: resolving contradictions between sections, eliminating redundancy, and constructing a coherent narrative from fragments.

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.xyc.ai/v1"  # any OpenAI-compatible endpoint
)

MAP_PROMPT = """Summarize the following document excerpt in 100 words or fewer.
Preserve key data points, conclusions, and proper nouns:

{chunk}

Summary:"""

REDUCE_PROMPT = """The following are summaries of individual sections from a long document.
Synthesize them into a single coherent summary (300 words or fewer).
Retain all key figures and conclusions, remove repetition, and organize logically:

{summaries}

Final summary:"""

async def map_chunk(chunk: str, semaphore: asyncio.Semaphore) -> str:
    async with semaphore:
        response = await client.chat.completions.create(
            model="claude-haiku-4-5",  # lightweight model for map phase
            messages=[{"role": "user", "content": MAP_PROMPT.format(chunk=chunk)}],
            max_tokens=300,
            temperature=0.3,
        )
        return response.choices[0].message.content

async def run_map_reduce(chunks: list[str], concurrency: int = 8) -> str:
    semaphore = asyncio.Semaphore(concurrency)  # throttle to avoid rate limit 429s

    # Map: process all chunks concurrently
    chunk_summaries = await asyncio.gather(
        *[map_chunk(chunk, semaphore) for chunk in chunks]
    )

    combined = "\n\n---\n\n".join(
        f"[Section {i+1}]\n{s}" for i, s in enumerate(chunk_summaries)
    )

    # Reduce: flagship model for final synthesis
    final_response = await client.chat.completions.create(
        model="claude-opus-4-8",
        messages=[{"role": "user", "content": REDUCE_PROMPT.format(summaries=combined)}],
        max_tokens=800,
        temperature=0.2,
    )
    return final_response.choices[0].message.content

concurrency=8 is a reasonable starting point for most APIs with ~60 RPM limits. At that rate, processing 50 chunks runs in about 40 seconds without triggering rate limit errors.

---

Hierarchical Reduce for Very Long Documents

When chunk count exceeds 50, the first reduce pass may itself overflow the context window. The fix is hierarchical reduction: group chunk summaries in batches of 10–15, reduce each group independently, then reduce the group summaries — repeat until you're down to one layer.

def hierarchical_reduce(summaries: list[str], group_size: int = 12) -> list[str]:
    """Recursively group and reduce until the summary list fits in one pass."""
    if len(summaries) <= group_size:
        return summaries

    groups = [summaries[i:i+group_size] for i in range(0, len(summaries), group_size)]
    reduced = [sync_reduce(group) for group in groups]  # async version follows same pattern
    return hierarchical_reduce(reduced, group_size)

---

Quality Control

A few practical checkpoints worth building into any production pipeline:

• Coverage check: Extract 5–10 key entities from the source — names, figures, dates — and verify each appears in the final summary.
• Hallucination detection: Ask the model to cite the source location in the original document for every number or claim in the final summary. Flag anything it can't locate.
• Length caps in the map prompt: Explicit word limits prevent one oversized chunk summary from bloating the reduce input.

For high-stakes documents — legal, medical, regulatory — it's worth adding an explicit instruction in the reduce prompt asking the model to distinguish between what the document directly states versus what it infers. That single prompt change reduces undetected confidence errors noticeably.

---

Model Selection and What It Actually Costs

For a typical 200-page PDF (~80K tokens), here's what the three main strategies cost per document:

Strategy	Map model	Reduce model	Estimated cost
All-flagship	Opus 4.8	Opus 4.8	~$1.60
Mixed (recommended)	Haiku 4.5	Opus 4.8	~$0.28
All-lightweight	Haiku 4.5	Haiku 4.5	~$0.06

The mixed strategy wins for most workloads. Map-phase tasks are simple enough for a lightweight model. The reduce phase — global synthesis, contradiction resolution, logical restructuring — is exactly where flagship reasoning pays off.

If you're running this pipeline through Claude Code, Codex, or Gemini CLI, the unified OpenAI-compatible format means you maintain one SDK and change a single parameter to switch models.

---

I've been running this pipeline on XycAi, which gives access to 200+ models — Claude, GPT-5.x, Gemini 3, and more — through a single OpenAI-compatible endpoint. The code above works as-is with just a base_url swap. For batch document processing in particular, the pricing (from 14% of official list price) makes the cost difference between strategies even more pronounced, and the CN2 direct-connect nodes keep latency low enough that high-concurrency runs stay stable.