Study guide

5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling

Claude Certified Developer - Foundations CCDV-F

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Core Messages API & Streaming

1.1 Messages API Fundamentals & Request Lifecycle 1.2 Streaming & Server-Sent Events (SSE) 1.3 Thinking: Adaptive Mode, Effort & Reasoning Budgets

2 Chapter 2: Optimization & Asynchronous Batches

2.1 Prompt Caching Mechanics & Breakpoints 2.2 Message Batches API & Asynchronous Workloads

3 Chapter 3: Structured Outputs, Documents & Files API

3.1 Structured Outputs & JSON Extraction 3.2 Files API, Document Blocks & Citations

4 Chapter 4: Application Design, Resilience & Delivery Life Cycle

4.1 API Resilience, Error Handling & Retries 4.2 Production Backend Architecture & State Management 4.3 Requirements Analysis & Claude Application Design 4.4 The Claude Application Life Cycle & Configuration Management

5 Chapter 5: LLM Fundamentals, Model Selection & Versioning

5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling 5.2 Claude Model Family Selection 5.3 Model Versioning, Snapshot Pinning & Migration 5.4 Latency & Throughput Optimization

6 Chapter 6: Cost Engineering & Context Management

6.1 Production Cost Optimization Strategies 6.2 Context Window Management & Compaction

7 Chapter 7: Foundations of Workflows & Deterministic Patterns

7.1 Agent vs. Workflow Architectural Foundations 7.2 Workflow Patterns: Prompt Chaining & Routing 7.3 Workflow Patterns: Parallelization & Voting

8 Chapter 8: Autonomous Agents & Multi-Step Systems

8.1 Workflow Patterns: Orchestrator-Workers & Evaluator-Optimizer 8.2 Agent Loop Implementation & State Tracking 8.3 Agent Frameworks & Human-in-the-Loop (HITL)

9 Chapter 9: Prompt and Context Engineering

9.1 System Prompt Design & Role Framing 9.2 XML Tags & Structural Prompt Delimiters 9.3 Few-Shot Prompting & In-Context Learning 9.4 RAG Context Injection & Document Formatting

10 Chapter 10: Tool Calling & Model Context Protocol (MCP)

10.1 Tool Schema Design & Parameter Validation 10.2 Tool Execution, Multi-Tool Calls & Error Handling 10.3 MCP Architecture & Core Primitives 10.4 MCP Transports & Client/Server Integration 10.5 Agentic Customization: Built-in Tools, Custom Tools, Skills & MCP

11 Chapter 11: Security, Safety & Governance

11.1 Direct & Indirect Prompt Injection Defenses 11.2 Tool Permissions, Sandboxing & Least Privilege 11.3 Data Privacy, Redaction & Audit Logging 11.4 Identity, Secrets & API Key Management

12 Chapter 12: Claude Code Developer Tooling

12.1 Claude Code Architecture & Project Configuration 12.2 Permissions, Modes & Hooks

13 Chapter 13: Evaluation, Testing & Production Observability

13.1 Evaluation Design & Golden Datasets 13.2 Production Observability, Tracing & Debugging

5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling

Key Takeaways Explain This Topic AI

A token averages about 4 characters or 0.75 English words, but code, JSON, non-Latin scripts, and long numbers are far denser, which is why /v1/messages/count_tokens exists rather than character estimation.

Claude 4.7 and later models use a newer tokenizer producing roughly 30% more tokens for the same text than Claude Sonnet 4.6 and earlier.

max_tokens is a ceiling rather than a reservation and is shared between thinking and response text; hitting it produces stop_reason max_tokens, which is truncation rather than completion.

Reproducibility comes from pinning the model snapshot and holding the prompt byte-identical, not from setting temperature to 0; current models (Claude 4.7 and later) reject a non-default temperature, top_p, or top_k with a 400 error.

The Claude API is REST over HTTPS; the API key goes in the Authorization header as a bearer token (the legacy x-api-key header still works), an anthropic-version header selects the API version, and streaming uses Server-Sent Events rather than WebSockets.

Last updated: October 2026

LLM & Technical Fundamentals: Tokens, Context Windows & Sampling

Exam Blueprint Focus: LLM Fundamentals (5.2%) and Technical Fundamentals (6.1%) are the two largest sub-skills in the 16.8% Model Selection and Optimisation domain — together they outweigh model selection and cost management combined. They test the layer below the API: what a token is, what a context window actually holds, what the sampling parameters do, and how the HTTP and SDK surface is shaped.

Tokens: The Unit You Are Billed In

A token is a sub-word fragment produced by a byte-pair-encoding (BPE) tokenizer. English text averages roughly 4 characters, or 0.75 words, per token , but that ratio is an average and not a rule:

  • Content Typical density Why
  • Ordinary English prose ~0.75 words/token Common words are single tokens
  • Source code Far more tokens per character Punctuation, indentation, and camelCase identifiers fragment
  • JSON Expensive Every brace, quote, colon, and comma is its own token
  • Non-Latin scripts Often 2-3x English Less represented in the tokenizer's merge table
  • Long numbers and UUIDs Very expensive Split into small digit groups

Two consequences engineers routinely get wrong:

You cannot estimate tokens from characters reliably , which is why /v1/messages/count_tokens exists. A 50,000-character English document and a 50,000-character JSON payload are not the same request.

Tokenizers change between model generations. Claude 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text than the one used by Claude Sonnet 4.6 and earlier. A prompt that fit a budget on one generation may not on the next, even byte-for-byte unchanged.

Where tokens are counted

Every one of these consumes context and is billed:

The system prompt, on every request.

The full messages array — the API is stateless , so the entire history is re-sent and re-billed each turn.

The tools array: names, descriptions, and JSON schemas.

An automatic tool-use system prompt the API injects whenever tools is present: 286 tokens on Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5 with tool_choice auto or none (406 on Claude Haiku 5.5 with any or tool ); the legacy Claude Sonnet 5 uses 354, rising to 474 when a tool is forced.

thinking blocks, billed as output tokens even when the text is not returned to you.

Context Windows: Capacity, Not Budget

The context window is the maximum number of tokens a single request may contain — input plus the output it generates. Current capacities: 1,000,000 tokens on every current model (Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5); 200,000 only on older models such as the legacy Claude Haiku 4.5 and Claude Sonnet 4.5. On the current tokenizer, 1M tokens is roughly 555,000 words.

Separately, max output caps the response: 128K tokens synchronously on every current model, and 64K on the legacy Claude Haiku 4.5. On the Message Batches API, several current models support up to 300K output tokens behind a beta header.

Three rules that follow:

max_tokens is a ceiling, not a reservation. You are billed for tokens actually generated. Setting it generously costs nothing directly — but if the model hits it, generation stops mid-sentence with stop_reason: "max_tokens" , which is a truncation bug, not a completion.

Thinking and response text share the max_tokens ceiling. At high effort , a small max_tokens starves the answer.

Capacity is not free. Every token in the window is re-billed on every turn it survives.

Sampling Parameters

After each forward pass the model produces a probability distribution over the vocabulary, and the next token is sampled from it. That is why generation is non-deterministic.

Parameter What it did Status on current models

temperature Flattened or sharpened the distribution Claude 4.7 and later: any non-default value returns a 400 error

top_p (nucleus) Kept only the smallest set of tokens whose cumulative probability reaches p Same as temperature

  • top_k Kept only the k highest-probability tokens Same as temperature
  • stop_sequences Halts generation when one is emitted; stop_reason becomes "stop_sequence" Supported

With API version 2026-10-21 , sending temperature , top_p , or top_k at all returns a 400 error on every model, even at the default value. Older models such as Claude Sonnet 4.6 still accept non-default values on 2023-06-01 .

Practical guidance the exam rewards:

Omit the sampling parameters. Anthropic's guidance is to remove them from request payloads and use prompting to guide the model's behavior instead.

Low temperature was never determinism. Even where it is accepted, identical output across runs is not guaranteed by the API. Reproducibility comes from pinning the model snapshot and holding the prompt byte-identical.

Structure beats sampling for structured output. If you need valid JSON, use structured outputs or a strict tool rather than lowering temperature and hoping. A schema enforces shape .

Technical Fundamentals: The HTTP and SDK Surface

Claude is a REST API over HTTPS . There is no WebSocket protocol and no session object on the server; streaming uses Server-Sent Events over ordinary HTTP , which matters for infrastructure design because SSE traverses standard HTTP proxies while WebSockets often do not. POST https://api.anthropic.com/v1/messages

  • Authorization: Bearer $ANTHROPIC_API_KEY
  • anthropic-version: 2026-10-21
  • content-type: application/json

Three headers, three facts:

Authorization: Bearer <key> is the documented way to send an API key. The legacy x-api-key: <key> header is still accepted in its place, which is why older examples use it.

anthropic-version pins the API contract independently of the model ( 2023-06-01 or 2026-10-21 ; the latest SDK major versions send 2026-10-21 ). Version pinning and model pinning are two different decisions.

anthropic-beta opts into pre-GA features; a request that needs a beta header and omits it fails rather than silently degrading.

The core endpoints:

  • Endpoint Purpose
  • POST /v1/messages Synchronous or streaming inference
  • POST /v1/messages/count_tokens Exact token count, no inference, no generation cost
  • POST /v1/messages/batches Asynchronous batch submission (50% discount)
  • POST /v1/files Upload a document once, reference it by ID
  • GET /v1/models Capabilities, max_input_tokens , max_tokens per model
  • What the SDKs add

The official SDKs ( anthropic for Python, @anthropic-ai/sdk for TypeScript, plus Go, Java, C#, Ruby, and PHP) are thin wrappers over that REST surface, and they are worth using because of what they handle for you: from anthropic import Anthropic

  • client = Anthropic() # reads ANTHROPIC_API_KEY from the environment
  • resp = client.messages.create(
  • model="claude-sonnet-5-5",
  • max_tokens=1024,
  • system="You are a precise technical assistant.",
  • messages=[{"role": "user", "content": "Summarise this incident report."}],
  • )

Automatic retries with exponential backoff on connection errors, 408, 409, 429, and 5xx responses (including 529), 2 retries by default, honoring retry-after when present.

Streaming helpers that reassemble SSE deltas — including partial JSON in tool_use blocks — into whole objects.

Typed response models , so a thinking block or a tool_use block is a typed object rather than a dict you must guess at.

Credential handling from the environment , so the key never has to appear in source.

Deployment surfaces

The same Messages API is reachable through Amazon Bedrock, Google Cloud (Vertex AI), Microsoft Foundry, and Claude Platform on AWS. The model IDs differ per platform, and partner-operated platforms set their own lifecycle dates and their own pricing — a fact that matters when you are tracking retirement across a multi-cloud deployment.

Loading diagram... Where Every Token in a Request Comes From