Study guide

2.2 Message Batches API & Asynchronous Workloads

Claude Certified Developer - Foundations CCDV-F

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Core Messages API & Streaming

1.1 Messages API Fundamentals & Request Lifecycle 1.2 Streaming & Server-Sent Events (SSE) 1.3 Thinking: Adaptive Mode, Effort & Reasoning Budgets

2 Chapter 2: Optimization & Asynchronous Batches

2.1 Prompt Caching Mechanics & Breakpoints 2.2 Message Batches API & Asynchronous Workloads

3 Chapter 3: Structured Outputs, Documents & Files API

3.1 Structured Outputs & JSON Extraction 3.2 Files API, Document Blocks & Citations

4 Chapter 4: Application Design, Resilience & Delivery Life Cycle

4.1 API Resilience, Error Handling & Retries 4.2 Production Backend Architecture & State Management 4.3 Requirements Analysis & Claude Application Design 4.4 The Claude Application Life Cycle & Configuration Management

5 Chapter 5: LLM Fundamentals, Model Selection & Versioning

5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling 5.2 Claude Model Family Selection 5.3 Model Versioning, Snapshot Pinning & Migration 5.4 Latency & Throughput Optimization

6 Chapter 6: Cost Engineering & Context Management

6.1 Production Cost Optimization Strategies 6.2 Context Window Management & Compaction

7 Chapter 7: Foundations of Workflows & Deterministic Patterns

7.1 Agent vs. Workflow Architectural Foundations 7.2 Workflow Patterns: Prompt Chaining & Routing 7.3 Workflow Patterns: Parallelization & Voting

8 Chapter 8: Autonomous Agents & Multi-Step Systems

8.1 Workflow Patterns: Orchestrator-Workers & Evaluator-Optimizer 8.2 Agent Loop Implementation & State Tracking 8.3 Agent Frameworks & Human-in-the-Loop (HITL)

9 Chapter 9: Prompt and Context Engineering

9.1 System Prompt Design & Role Framing 9.2 XML Tags & Structural Prompt Delimiters 9.3 Few-Shot Prompting & In-Context Learning 9.4 RAG Context Injection & Document Formatting

10 Chapter 10: Tool Calling & Model Context Protocol (MCP)

10.1 Tool Schema Design & Parameter Validation 10.2 Tool Execution, Multi-Tool Calls & Error Handling 10.3 MCP Architecture & Core Primitives 10.4 MCP Transports & Client/Server Integration 10.5 Agentic Customization: Built-in Tools, Custom Tools, Skills & MCP

11 Chapter 11: Security, Safety & Governance

11.1 Direct & Indirect Prompt Injection Defenses 11.2 Tool Permissions, Sandboxing & Least Privilege 11.3 Data Privacy, Redaction & Audit Logging 11.4 Identity, Secrets & API Key Management

12 Chapter 12: Claude Code Developer Tooling

12.1 Claude Code Architecture & Project Configuration 12.2 Permissions, Modes & Hooks

13 Chapter 13: Evaluation, Testing & Production Observability

13.1 Evaluation Design & Golden Datasets 13.2 Production Observability, Tracing & Debugging

2.2 Message Batches API & Asynchronous Workloads

Key Takeaways Explain This Topic AI

The Message Batches API provides a 50% discount on all token types (input, output, cache write, and cache read); most batches finish in under an hour and any request not processed within 24 hours expires unbilled.

Each request inside a batch requires a unique custom_id of 1 to 64 alphanumeric, hyphen, or underscore characters, which developers must use to correlate results because lines in the output JSONL file are non-deterministic in order.

A batch is limited to 100,000 Message requests or 256 MB, whichever is reached first, and batch requests draw on a rate-limit pool separate from synchronous interactive traffic.

Error boundaries operate at individual item granularity: failures (validation, rate limit, timeout) are reported per custom_id without aborting the broader batch execution.

Canceling a batch halts pending items while allowing currently running requests to complete and yield downloadable partial results.

Last updated: October 2026

2.2 Message Batches API & Asynchronous Workloads

Core Concept: The Anthropic Message Batches API provides an asynchronous, high-throughput execution channel engineered for bulk, non-interactive workloads. By decoupling requests from real-time synchronous serving constraints, the Batches API provides a flat 50% discount across all token modalities (input, output, cache creation, and cache read) with most batches finishing in under 1 hour and a hard 24-hour expiration .

Architectural Purpose of the Message Batches API

Modern enterprise AI architectures frequently require processing massive volumes of data where immediate sub-second responses are unnecessary. Running millions of tokens through standard synchronous endpoints ( POST /v1/messages ) introduces significant architectural friction:

Connection Fragility: Synchronous HTTP connections held open across long generation windows risk timeout errors, dropped sockets, and transient network disconnects.

Rate Limit Contention: High-volume batch jobs (such as re-indexing document embeddings, evaluating historical conversations, or running overnight classification pipelines) compete directly with user-facing interactive traffic for requests-per-minute (RPM), input-tokens-per-minute (ITPM), and output-tokens-per-minute (OTPM) limits.

Economic Inefficiency: Paying standard retail token pricing for background tasks that could easily execute during off-peak inference windows wastes organizational budget.

The Message Batches API solves these challenges by providing a dedicated asynchronous ingestion and execution pipeline. In exchange for relaxing the latency guarantee - most batches finish within an hour, and any request not sent to the model within 24 hours expires - Anthropic cuts all token costs in half—including inputs, outputs, prompt cache writes, and prompt cache reads. Furthermore, batches execute against dedicated asynchronous capacity pools, shielding interactive production services from rate limit exhaustion.

Synchronous vs. Asynchronous Decision Matrix

Selecting between the standard Messages API (with or without streaming) and the Message Batches API is a foundational architectural decision:

Evaluation Dimension Standard Messages API (Synchronous) Message Batches API (Asynchronous)

Turnaround Latency Immediate; streaming shows the first token as soon as it is generated Asynchronous; most batches under 1 hour, hard 24-hour expiration

Pricing Structure 100% standard retail token pricing 50% discount on all input, output, and cached tokens

Connection Model Persistent client-server connection (HTTP request/response or SSE) Asynchronous: Submit -> Poll/Webhook -> Download JSONL

Concurrency & Quotas Shared real-time RPM / ITPM / OTPM rate limits Separate asynchronous capacity; up to 100,000 requests or 256 MB per batch

Error Isolation Request-level; single connection failure drops the call Per-request boundary; individual item failure does not halt batch

Prompt Caching Support Fully supported Fully supported; 50% discount stacks with cache savings

Ideal Workloads Interactive chat, coding assistants, low-latency agent loops Evals, document classification, synthetic data, overnight backfills

Batch Request Lifecycle & Operations

The Message Batches API follows a decoupled lifecycle consisting of submission, background processing, monitoring, and result retrieval.

1. Creating a Batch (POST /v1/messages/batches)

Batches are initialized by posting a collection of message requests. Each individual item in the requests list requires two primary attributes:

custom_id : A developer-assigned unique string (1 to 64 characters; alphanumeric, underscores, and hyphens) used to correlate responses back to original records.

params : A standard Messages API payload object specifying model , max_tokens , messages , and optional configurations like system , tools , output_config , or cache_control (current models reject non-default sampling parameters such as temperature in batch requests just as in synchronous ones). import anthropic

client = anthropic.Anthropic()

  • batch = client.messages.batches.create(
  • requests=[
  • {
  • "custom_id": f"doc-classification-{idx}",
  • "params": {
  • "model": "claude-sonnet-5-5",
  • "max_tokens": 256,
  • "system": [
  • {
  • "type": "text",
  • "text": "Classify the sentiment and category of the provided legal filing.",

"cache_control": {"type": "ephemeral"} # Caching works in batches!

  • }
  • ],
  • "messages": [
  • {"role": "user", "content": document_text}
  • ]
  • }
  • }
  • for idx, document_text in enumerate(corpus_documents)
  • ]
  • )
  • print(f"Batch created with ID: {batch.id}, Status: {batch.processing_status}")
  • 2. Batch Lifecycle States

A batch transitions through distinct states recorded in processing_status :

in_progress : The batch has been validated and queued. Requests are executing concurrently across worker pools.

canceling : A cancellation command has been submitted via POST /v1/messages/batches/{batch_id}/cancel . Workers are terminating un-started tasks while allowing running tasks to finish.

ended : Terminal state. All requests in the batch have reached completion ( succeeded , errored , canceled , or expired ).

3. Monitoring Batch Progress

Clients query batch state via GET /v1/messages/batches/{batch_id} . The response provides fine-grained counters in the request_counts object: {

  • "id": "msgbatch_01ABCxyz...",
  • "type": "message_batch",
  • "processing_status": "ended",
  • "request_counts": {
  • "processing": 0,
  • "succeeded": 9982,
  • "errored": 18,
  • "canceled": 0,
  • "expired": 0
  • },
  • "created_at": "2026-09-10T02:00:00Z",
  • "ended_at": "2026-09-10T02:45:12Z",
  • "results_url": "https://anthropic-batches.s3.amazonaws.com/results_01ABCxyz.jsonl?AWSAccessKeyId=..."
  • }
  • Result Processing & Result Correlation via custom_id

Once a batch enters the ended state, results are available for download as a newline-delimited JSON (JSONL) file.

The Non-Deterministic Ordering Rule

A fundamental architectural characteristic of the Batches API is that results in the JSONL output file are NOT emitted in the original submission order .

Because Anthropic's distributed scheduler processes requests asynchronously across thousands of parallel worker nodes, shorter prompts or faster completions may finish ahead of earlier submissions.

Developers must correlate results using custom_id rather than array indexing.

JSONL Schema Structure

Each line in the downloaded JSONL file contains an independent JSON object encapsulating the result of a single custom_id : {"custom_id": "doc-classification-0", "result": {"type": "succeeded", "message": {"id": "msg_01...", "type": "message", "role": "assistant", "content": [{"type": "text", "text": "Category: Regulatory Compliance. Sentiment: Neutral."}], "model": "claude-sonnet-5-5", "stop_reason": "end_turn", "usage": {"input_tokens": 1520, "output_tokens": 18, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 1450}}}}

{"custom_id": "doc-classification-1", "result": {"type": "errored", "error": {"type": "invalid_request_error", "message": "max_tokens: 256 is smaller than minimum required"}}}

  • {"custom_id": "doc-classification-2", "result": {"type": "canceled"}}
  • {"custom_id": "doc-classification-3", "result": {"type": "expired"}}
  • Result Status Types & Error Boundaries

The result.type field reflects one of four outcomes:

"succeeded" : The request completed normally. The object contains the standard Anthropic message object with full content , usage , and token metrics.

"errored" : The individual request failed (e.g., malformed payload, invalid schema, or content filter trigger). It contains an error object with standard error codes.

"canceled" : The batch was canceled by the client before this specific request began processing.

"expired" : The batch reached its 24-hour expiration before this request was sent to the model. You are billed for neither expired nor errored requests.

Granular Error Isolation

In synchronous API calls, a batch of requests orchestrated via client-side threads can fail unpredictably if connection drops or rate limits hit. In the Batches API, error boundaries are isolated to the individual request . If 18 out of 10,000 requests in a batch contain an invalid parameter, those 18 requests emit "errored" lines in the JSONL output, while the remaining 9,982 requests process to "succeeded" status without interruption.

  • Quotas, Limits, and Cancellation Semantics
  • Architectural Limits

Maximum Requests per Batch: 100,000 Message requests .

Maximum Batch Payload Size: 256 MB . A batch is capped by whichever of the two ceilings it reaches first, so work beyond either bound must be partitioned across batches.

Result Retention Window: Results remain available for download for 29 days after the batch is created (counted from created_at , not from when processing ends). After that you can still view the batch, but its results can no longer be downloaded.

Processing Window: Most batches complete in under 1 hour . Results become available once every request finishes or after 24 hours, whichever comes first; any request still unprocessed at 24 hours expires and is not billed. Actual speed depends on cluster load and queue depth.

Cancellation Semantics

A batch in the in_progress state can be canceled at any time by issuing: POST /v1/messages/batches/{batch_id}/cancel

Understanding cancellation behavior is a common exam topic:

The batch immediately transitions from in_progress to canceling .

Any individual requests that are currently executing in worker memory will run to completion and be recorded as "succeeded" (or "errored" ).

Requests that are still queued and have not yet started execution are halted and recorded as "canceled" .

Once all active requests finalize, the batch transitions to ended .

Developers can still download the partial results file via results_url to retrieve all requests that succeeded before the cancellation completed.

Combining Prompt Caching with Message Batches

A powerful production optimization pattern is compounding Prompt Caching with the Message Batches API .

Because prompt caching operates inside batch processing:

The 50% batch discount applies to all token types , including cache_creation_input_tokens and cache_read_input_tokens .

For Claude Sonnet 5.5:

  • Synchronous base input: $2.00 / MTok
  • Batch base input (50% off): $1.00 / MTok
  • Synchronous cache read (0.05x on this model): $0.10 / MTok
  • Batch cache read (50% of 0.05x): $0.05 / MTok

When evaluating large document corpuses against a static instruction set using batching and prompt caching together, the cached input portion costs 97.5% less than standard synchronous un-cached input on Claude Sonnet 5.5 ($0.05 vs $2.00 per MTok). On a model with the standard 0.1x read multiplier, such as Claude Haiku 5.5 or the legacy Claude Sonnet 5, the same combination saves 95%.

Loading diagram... Message Batches API Lifecycle and Processing Pipeline