Study guide

6.2 Context Window Management & Compaction

Claude Certified Developer - Foundations CCDV-F

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Core Messages API & Streaming

1.1 Messages API Fundamentals & Request Lifecycle 1.2 Streaming & Server-Sent Events (SSE) 1.3 Thinking: Adaptive Mode, Effort & Reasoning Budgets

2 Chapter 2: Optimization & Asynchronous Batches

2.1 Prompt Caching Mechanics & Breakpoints 2.2 Message Batches API & Asynchronous Workloads

3 Chapter 3: Structured Outputs, Documents & Files API

3.1 Structured Outputs & JSON Extraction 3.2 Files API, Document Blocks & Citations

4 Chapter 4: Application Design, Resilience & Delivery Life Cycle

4.1 API Resilience, Error Handling & Retries 4.2 Production Backend Architecture & State Management 4.3 Requirements Analysis & Claude Application Design 4.4 The Claude Application Life Cycle & Configuration Management

5 Chapter 5: LLM Fundamentals, Model Selection & Versioning

5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling 5.2 Claude Model Family Selection 5.3 Model Versioning, Snapshot Pinning & Migration 5.4 Latency & Throughput Optimization

6 Chapter 6: Cost Engineering & Context Management

6.1 Production Cost Optimization Strategies 6.2 Context Window Management & Compaction

7 Chapter 7: Foundations of Workflows & Deterministic Patterns

7.1 Agent vs. Workflow Architectural Foundations 7.2 Workflow Patterns: Prompt Chaining & Routing 7.3 Workflow Patterns: Parallelization & Voting

8 Chapter 8: Autonomous Agents & Multi-Step Systems

8.1 Workflow Patterns: Orchestrator-Workers & Evaluator-Optimizer 8.2 Agent Loop Implementation & State Tracking 8.3 Agent Frameworks & Human-in-the-Loop (HITL)

9 Chapter 9: Prompt and Context Engineering

9.1 System Prompt Design & Role Framing 9.2 XML Tags & Structural Prompt Delimiters 9.3 Few-Shot Prompting & In-Context Learning 9.4 RAG Context Injection & Document Formatting

10 Chapter 10: Tool Calling & Model Context Protocol (MCP)

10.1 Tool Schema Design & Parameter Validation 10.2 Tool Execution, Multi-Tool Calls & Error Handling 10.3 MCP Architecture & Core Primitives 10.4 MCP Transports & Client/Server Integration 10.5 Agentic Customization: Built-in Tools, Custom Tools, Skills & MCP

11 Chapter 11: Security, Safety & Governance

11.1 Direct & Indirect Prompt Injection Defenses 11.2 Tool Permissions, Sandboxing & Least Privilege 11.3 Data Privacy, Redaction & Audit Logging 11.4 Identity, Secrets & API Key Management

12 Chapter 12: Claude Code Developer Tooling

12.1 Claude Code Architecture & Project Configuration 12.2 Permissions, Modes & Hooks

13 Chapter 13: Evaluation, Testing & Production Observability

13.1 Evaluation Design & Golden Datasets 13.2 Production Observability, Tracing & Debugging

6.2 Context Window Management & Compaction

Key Takeaways Explain This Topic AI

Every current model (Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5) carries a 1,000,000-token context window; 200,000 tokens applies only to older models such as the legacy Claude Haiku 4.5, and only Haiku 5.5 charges more for prompts over 100,000 tokens.

Context capacity is a ceiling rather than a budget: every surviving token is re-billed on every turn, prefill latency scales with input length, and retrieval accuracy degrades in the middle of a long context.

The 'lost in the middle' effect produces a U-shaped attention curve in which content at the start and end of the context is recalled far more reliably than content buried in the middle.

For long-running conversations and agents, Anthropic's primary strategy is server-side compaction (Claude writes the summary on the server), with context editing (clear_tool_uses_20250919) to clear old tool results; client-side sliding windows are cheap but amnesiac, and recursive summarization or a tiered layout preserves gist while losing verbatim detail.

Raw tool output must be compacted before it enters the agent's context, using schema projection, in-flight summarization by a cheaper model, or reference handles with pagination.

Last updated: October 2026

Context Window Management & Compaction

Exam Blueprint Focus: The Claude Certified Developer - Foundations exam tests your understanding of context window dynamics, including attention degradation ("Lost in the Middle"), server-side compaction and context editing, multi-turn memory architectures, tool output compaction, and client-side patterns such as a tiered context layout. Candidates must understand how to maintain semantic fidelity while optimizing token efficiency across long-running conversational interactions and autonomous agents.

The 1M-Token Window: Operational Capacity vs. Cognitive Budget

Every current model — Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5 — carries a 1,000,000-token context window , and 1M is the default with no beta header. Long context is billed at standard per-token pricing on these models, except Claude Haiku 5.5, where prompts over 100,000 tokens move to a higher price band. Older models such as the legacy Claude Haiku 4.5 remain at 200,000 tokens , so routing to one of them still lowers your ceiling.

A 1M-token window is roughly 555,000 words on the current tokenizer (introduced with Claude Opus 4.7); models before it fit about 750,000 words in the same 1M tokens. A 200K window is roughly 150,000 words. Either way the operational capacity is enormous: entire codebases, multi-hundred-page filings, or hours of transcript.

The trap is treating that capacity as a budget to spend rather than a ceiling not to hit . Capacity is not the same as attention, and it is certainly not the same as cost. Three separate constraints bind long before the window does:

Cost. Every token in the window is billed on every turn it survives. A 400,000-token context on Claude Sonnet 5.5 costs $0.80 in input per request before caching, and a 40-turn session repeats that charge 40 times.

Latency. Prefill scales with input length. A window you fill is a TTFT you pay.

Attention quality. Retrieval accuracy is not uniform across a long context — which is the subject of the rest of this section.

The Architectural Trap: Unchecked Context Accumulation

While a 1M-token context window lets developers ingest massive documents, relying on full context accumulation in multi-turn conversations is an architectural anti-pattern. Two severe challenges arise when contexts expand unchecked:

Compounding Economic Cost: In Anthropic's stateless API architecture, the client must submit the entire conversational transcript on every turn. If a customer conversation accumulates 100,000 tokens of dialogue and tool outputs:

Turn 1: 5,000 input tokens.

Turn 5: 35,000 input tokens.

Turn 15: 100,000 input tokens.

On Claude Sonnet 5.5 ($2.00/MTok base input), submitting 100,000 tokens of un-cached input costs $0.20 per single request . Across 10 subsequent turns, the developer pays $2.00 in input tokens alone for a single user session!

Time-to-First-Token (TTFT) Latency: Prompt prefill latency scales with input token volume, so a very large uncached prompt delays the first output token and degrades real-time user experiences.

Attention Degradation Dynamics: "Lost in the Middle"

In addition to cost and latency overhead, massive context windows introduce cognitive and retrieval degradation known in transformer research as the "Lost in the Middle" phenomenon .

Theoretical Mechanics: Attention Dilution in Transformer Layers

To understand why models struggle with massive contexts, we must examine the mathematical foundation of transformer self-attention: A t t e n t i o n ( Q , K , V ) = s o f t m a x ( Q K T d k ) V \mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V Attention ( Q , K , V ) = softmax ( d k ​ ​ Q K T ​ ) V

During the attention calculation, query vector q i q_i q i ​ computes dot products with key vectors k j k_j k j ​ across all N N N tokens in the sequence. These scalar affinities are normalized via the softmax function: α i j = exp ⁡ ( q i k j T / d k ) ∑ m = 1 N exp ⁡ ( q i k m T / d k ) \alpha_{ij} = \frac{\exp(q_i k_j^T / \sqrt{d_k})}{\sum_{m=1}^N \exp(q_i k_m^T / \sqrt{d_k})} α ij ​ = ∑ m = 1 N ​ exp ( q i ​ k m T ​ / d k ​ ​ ) exp ( q i ​ k j T ​ / d k ​ ​ ) ​

As sequence length N N N expands from 2,000 tokens to 150,000 tokens:

The denominator ∑ m = 1 N exp ⁡ ( q i k m T / d k ) \sum_{m=1}^N \exp(q_i k_m^T / \sqrt{d_k}) ∑ m = 1 N ​ exp ( q i ​ k m T ​ / d k ​ ​ ) aggregates affinity values across tens of thousands of tokens.

Even if key tokens have relatively strong semantic affinity, the normalization over thousands of irrelevant or background tokens inevitably dilutes the peak attention probability mass .

Softmax temperature distribution creates entropy spread, making it harder for attention heads to resolve subtle, isolated facts buried in dense narrative context.

The U-Shaped Attention Curve

Published research on long-context language models (the "Lost in the Middle" study) found that retrieval accuracy often follows a U-shaped curve : best for content at the start and end of the context, worse in the middle. The exact shape varies by model and task, so measure it on your own workload: Attention / Retrieval

  • Fidelity
  • 100% ──┐ ┌── 100%
  • │ PRIMACY EFFECT │ RECENCY EFFECT
  • │ (System Prompts, │ (Recent User Query,
  • │ Core Directives) │ Immediate History)
  • 50% ──┤ ├── 50%
  • │ VALLEY OF INATTENTION │
  • │ (middle of the context) │
  • 0% ──┴───────────────\_____________________________/───────┴── 0%
  • Token 0 Token N
  • (Start of Prompt) (End of Prompt)

Primacy Effect (start of the context): Models tend to recall content positioned at the very beginning of the prompt well. System instructions, persona guidelines, and primary constraints placed here maintain strong steering influence.

Recency Effect (end of the context): Models tend to use the tail of the prompt well. Anthropic's own long-context guidance follows from this: put long documents near the top and the query at the end. The latest user message, active tool execution results, and formatting constraints placed at the end are processed with high accuracy.

The "Valley of Inattention" (middle of the context): Information positioned in the middle of a massive context window is the most likely to be missed. Subtle facts, negative constraints, or reference data located in this zone suffer higher omission rates, hallucination risk, and instruction drift.

Multi-Turn Conversation Management Strategies

To counteract attention dilution and compounding costs, production systems need structured context management. Start with what the API already provides, then add client-side patterns where you need more control.

0. Anthropic's Server-Side Options: Compaction and Context Editing

Server-side compaction (beta) is Anthropic's primary strategy for long-running conversations and agentic workflows. Compaction replaces the older turns of a conversation with a summary that Claude writes on the server, so you need no summarization code of your own. It comes in two forms: compaction on demand (you request the summary and swap the returned block in; it can keep recent turns word for word or run in the background) and compaction at a token threshold (the API compacts inside an ordinary request when input reaches the trigger you set). Send the returned compaction block back as instructed; dropping it loses the compacted history.

Context editing (beta, context-management-2025-06-27 ) clears content by rule instead of summarizing it. The clear_tool_uses_20250919 strategy clears the oldest tool results once context grows beyond your configured threshold, replacing each with placeholder text; clear_thinking_20251015 clears thinking blocks from earlier turns.

The client-side patterns below remain useful when you need full control over what survives, or on models and platforms where the server-side features are not available. One current-model caution: on Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, a thinking block stays valid only while everything before it is unchanged, so a pattern that summarizes, drops, or shortens earlier turns in place invalidates the later thinking blocks (a 400 error by default for accounts created on or after 31 August 2026). Shorten tool output before its first send, keep the history append-only, and use server-side compaction or context editing for trimming:

1. Sliding Window (FIFO Truncation)

The Sliding Window strategy maintains a fixed budget of recent dialogue turns, discarding the oldest turns as new messages arrive.

Mechanism: The system retains the static system prompt plus the most recent K K K user-assistant message pairs (such as the last 6 turns). When turn K + 1 K+1 K + 1 arrives, turn 1 is permanently dropped from the payload.

Advantages: Trivial to implement; strictly bounds input token volume and latency to a deterministic upper ceiling; zero additional LLM computation or API calls required.

Disadvantages: Suffers from catastrophic amnesia . If a user states critical personal constraints, preferences, or requirements in turn 2 (such as "I have a severe peanut allergy" or "My customer ID is ACC-9842" ), that information is completely forgotten once the conversation advances past the window threshold.

2. Structured Recursive Summarization

The Structured Recursive Summarization strategy periodically compresses historical dialogue turns into a concise narrative summary block that is injected into subsequent turns.

Mechanism: The system monitors cumulative conversation tokens. When the transcript exceeds a defined watermark (such as 6,000 tokens), an asynchronous background task is triggered.

The background worker invokes a fast, inexpensive model (Claude Haiku 5.5) with a specialized summarization prompt.

Historical turns (for example, turns 1 through 10) are condensed into an XML-wrapped summary block: <conversation_summary> .

The primary conversation history is then reconstructed as: [System Prompt] + [<conversation_summary>] + [Recent 4 Turns] .

Advantages: Retains long-term narrative continuity and user decisions across dozens of turns; compresses 10,000 tokens of dialogue into 300 tokens (a 97% context reduction ); cost overhead is negligible when using Claude Haiku 5.5 (from $0.10/MTok input).

Disadvantages: Compaction inevitably loses verbatim phrasing and subtle conversational tone; asynchronous background summarization requires state tracking to avoid race conditions.

3. A Tiered Context Layout (Client-Side Pattern)

A tiered context layout is a common client-side design (not an Anthropic-defined standard). Rather than treating conversation history as a monolithic flat array, it segregates context into three functional tiers: +------------------------------------------------------------------------+

  • | TIER 1: Immutable Core Context (Cached via cache_control) |
  • | - System prompt, core operational rules, safety guardrails |
  • | - Comprehensive Tool Definitions (OpenAPI schemas) |
  • | - Static User Identity Profile & Account Metadata |
  • +------------------------------------------------------------------------+
  • | TIER 2: Dynamic Entity & State Scratchpad (Structured XML) |
  • | - Active Goal State & Progress Checklist |
  • | - Verified Key-Value Entities (Account numbers, constraints, decisions)|
  • | - Condensed Historical Dialogue Summary |
  • +------------------------------------------------------------------------+
  • | TIER 3: Rolling Dialogue Window (Immediate Context) |
  • | - Most recent 4 to 6 raw user and assistant turns |
  • | - Immediate conversational fluency, references, and sentiment |
  • +------------------------------------------------------------------------+
  • Why Tiered Architecture Outperforms Flat Contexts

Maximizes Prompt Caching: Tier 1 is completely static across the entire user session. Placing a cache_control: {"type": "ephemeral"} breakpoint at the end of Tier 1 bills the core instructions at the cache-read rate on every hit (0.1x base input on most models, 0.05x on Claude Opus 5.5 and Claude Sonnet 5.5, 0.025x on Claude Fable 5.1).

Prevents "Lost in the Middle": By distilling older turns into an explicit structured state scratchpad (Tier 2), key user constraints are kept concise and placed directly alongside active goals rather than lingering in a 50,000-token conversational graveyard.

Preserves Conversational Fluency: Tier 3 provides raw verbatim context for the most recent exchanges, allowing Claude to understand pronouns, conversational references, and immediate conversational rhythm.

Handling Long Documents & Tool Outputs in Autonomous Agents

In autonomous agentic architectures (such as ReAct loops, Orchestrator-Worker systems, or coding agents), the single greatest source of context bloat is uncompacted tool outputs .

The Tool Output Explosion Problem

Consider an autonomous financial agent equipped with a query_database tool:

The agent calls query_database("SELECT * FROM customer_transactions WHERE year = 2025") .

The database returns an unformatted JSON array containing 1,000 rows of transaction data—consuming 18,000 tokens .

The agent receives this 18,000-token string inside a tool_result content block.

The agent requires 5 subsequent tool calls to analyze anomalies, cross-reference account balances, and format the final audit.

In a naive stateless implementation, that single 18,000-token tool result is re-transmitted on every subsequent turn: 18 , 000 tokens × 5 remaining turns = 90 , 000 redundant input tokens! 18,000\text{ tokens} \times 5\text{ remaining turns} = 90,000\text{ redundant input tokens!} 18 , 000 tokens × 5 remaining turns = 90 , 000 redundant input tokens!

On Claude Sonnet 5.5 ($2.00/MTok base input), this single uncompacted tool call burns about $0.18 in redundant input costs (90,000 × $2.00 / 1,000,000) before caching and injects 18,000 tokens of noisy tabular data directly into the middle of the context, severely elevating the risk of attention dilution and hallucination.

  • Production Compaction Patterns for Agentic Tools
  • Pattern 1: Schema Projection and Client-Side Filtering

Never return raw database rows or full third-party API envelopes into Claude's context. The client-side tool execution handler should project only the specific fields requested by the prompt:

Strip internal database IDs, foreign key UUIDs, timestamps, and metadata headers.

Transform deeply nested JSON into concise tabular text or CSV representations, which usually consume fewer tokens than verbose JSON (measure with count_tokens ).

Pattern 2: In-Flight Tool Output Summarization (Worker Model Pattern)

When a tool returns an unavoidably massive payload (such as a 40-page PDF, a 500-line server error trace, or a 10,000-row dataset):

Do not inject the raw payload directly into the orchestrator agent's conversation history.

Route the raw payload to an isolated, fast worker model (Claude Haiku 5.5) with an extraction prompt: "Extract the top 5 transaction anomalies, total expenditure sum, and flagged merchant IDs from this raw dataset. Return a structured 200-token XML summary."

Inject the resulting 200-token summary into the orchestrator's tool_result block. # In-Flight Compaction Handler inside Agent Loop

def execute_database_tool(query: str) -> str:

raw_results = db.execute(query) # Returns 15,000 tokens of raw SQL records

# Check if payload exceeds safe context injection watermark

if estimate_tokens(raw_results) > 1500:

  • # Compact payload using lightweight Claude Haiku 5.5 worker
  • summary_response = haiku_client.messages.create(
  • model="claude-haiku-5-5",
  • max_tokens=400,
  • thinking={"type": "disabled"}, # small worker; thinking would share the 400 tokens

system="You are a data compaction worker. Distill raw database rows into key statistical metrics, trends, and notable outliers.",

  • messages=[{"role": "user", "content": f"Summarize these records for downstream analysis:\n{raw_results}"}]
  • )
  • return next(b.text for b in summary_response.content if b.type == "text")
  • return raw_results
  • Pattern 3: Reference Handles & Pagination

For extremely large datasets, store the full artifact in an external object store (such as AWS S3, Redis, or local disk) and return an opaque reference handle along with summary statistics: {

  • "artifact_id": "art_9842_transactions_q3",
  • "total_records": 4820,
  • "summary_metrics": {"total_spend": "$1,420,800.00", "flagged_count": 12},
  • "sample_records": [
  • {"tx_id": "tx_101", "merchant": "Acme Cloud", "amount": 42500.00, "flag": "outlier"}
  • ],

"note": "Full records stored in art_9842. Use fetch_record_details(artifact_id, tx_id) to inspect specific items."

}

This pattern allows the agent to inspect overview metrics and surgically query specific items on demand, maintaining a lean context window.

Comparative Architecture Matrix: Context Management Patterns

Pattern Token Overhead Risk of Amnesia Implementation Complexity Prompt Caching Synergy Best Production Use Case

Full Accumulation Extreme (O( N 2 N^2 N 2 ) token growth) None (Verbatim history preserved) Trivial (Append all messages) Low (Dynamic turns invalidate cache tail) Short single-session tasks (< 4 turns)

Sliding Window Minimal (Strictly bounded) High (Catastrophic amnesia beyond K K K turns) Very Low (FIFO slice on messages array) Moderate (Can cache system prompt) Simple ephemeral customer chats, basic FAQ bots

Structured Summarization Low (Compresses history ~90%) Low (Key decisions retained in summary) Medium (Requires background Haiku worker) Moderate (Summary updates alter prompt prefix) Multi-turn customer service sessions, personal assistants

Server-Side Compaction Low (Claude summarizes older turns on the server) Low to medium (summary loses verbatim detail; recent turns can be kept) Low (beta parameter; send the compaction block back) Moderate (the compacted prefix changes when compaction runs) Long-running conversations and agents (Anthropic's primary strategy)

Tiered Context Layout Optimized (Predictable, controlled growth) Very Low (Core facts in scratchpad, dialogue in window) High (Requires state extraction & entity store) Maximum (Tier 1 permanently cached; Tier 2/3 isolated) Enterprise autonomous agents, complex long-running advisory systems

Loading diagram... Tiered Context Architecture & Compaction Lifecycle