Study guide
13.2 Production Observability, Tracing & Debugging
Claude Certified Developer - Foundations CCDV-F
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Core Messages API & Streaming
1.1 Messages API Fundamentals & Request Lifecycle 1.2 Streaming & Server-Sent Events (SSE) 1.3 Thinking: Adaptive Mode, Effort & Reasoning Budgets
2 Chapter 2: Optimization & Asynchronous Batches
2.1 Prompt Caching Mechanics & Breakpoints 2.2 Message Batches API & Asynchronous Workloads
3 Chapter 3: Structured Outputs, Documents & Files API
3.1 Structured Outputs & JSON Extraction 3.2 Files API, Document Blocks & Citations
4 Chapter 4: Application Design, Resilience & Delivery Life Cycle
4.1 API Resilience, Error Handling & Retries 4.2 Production Backend Architecture & State Management 4.3 Requirements Analysis & Claude Application Design 4.4 The Claude Application Life Cycle & Configuration Management
5 Chapter 5: LLM Fundamentals, Model Selection & Versioning
5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling 5.2 Claude Model Family Selection 5.3 Model Versioning, Snapshot Pinning & Migration 5.4 Latency & Throughput Optimization
6 Chapter 6: Cost Engineering & Context Management
6.1 Production Cost Optimization Strategies 6.2 Context Window Management & Compaction
7 Chapter 7: Foundations of Workflows & Deterministic Patterns
7.1 Agent vs. Workflow Architectural Foundations 7.2 Workflow Patterns: Prompt Chaining & Routing 7.3 Workflow Patterns: Parallelization & Voting
8 Chapter 8: Autonomous Agents & Multi-Step Systems
8.1 Workflow Patterns: Orchestrator-Workers & Evaluator-Optimizer 8.2 Agent Loop Implementation & State Tracking 8.3 Agent Frameworks & Human-in-the-Loop (HITL)
9 Chapter 9: Prompt and Context Engineering
9.1 System Prompt Design & Role Framing 9.2 XML Tags & Structural Prompt Delimiters 9.3 Few-Shot Prompting & In-Context Learning 9.4 RAG Context Injection & Document Formatting
10 Chapter 10: Tool Calling & Model Context Protocol (MCP)
10.1 Tool Schema Design & Parameter Validation 10.2 Tool Execution, Multi-Tool Calls & Error Handling 10.3 MCP Architecture & Core Primitives 10.4 MCP Transports & Client/Server Integration 10.5 Agentic Customization: Built-in Tools, Custom Tools, Skills & MCP
11 Chapter 11: Security, Safety & Governance
11.1 Direct & Indirect Prompt Injection Defenses 11.2 Tool Permissions, Sandboxing & Least Privilege 11.3 Data Privacy, Redaction & Audit Logging 11.4 Identity, Secrets & API Key Management
12 Chapter 12: Claude Code Developer Tooling
12.1 Claude Code Architecture & Project Configuration 12.2 Permissions, Modes & Hooks
13 Chapter 13: Evaluation, Testing & Production Observability
13.1 Evaluation Design & Golden Datasets 13.2 Production Observability, Tracing & Debugging
13.2 Production Observability, Tracing & Debugging
Key Takeaways Explain This Topic AI
Production LLM observability requires monitoring semantic metrics (token economics, prompt cache hit ratios, stop reason distributions, and tool error rates) in addition to traditional infrastructure metrics.
Latency telemetry must distinguish between Time-to-First-Token (TTFT), which reflects prompt compilation and cache retrieval, and Inter-Token Latency (ITL), which measures streaming token emission throughput.
OpenTelemetry distributed tracing decomposes complex multi-step agent interactions into parent-child spans covering user prompt ingestion, routing classifications, model inference, tool execution, and client response streaming.
Production failure modes—such as infinite agent loops, tool parameter hallucination, unexpected max_tokens truncation, and dynamic timestamp cache invalidation—require distinct programmatic guardrails and alert thresholds.
Replay debugging relies on capturing the exact array of messages, tool definitions, system prompt, model ID, and request parameters such as effort to reproduce production edge cases locally; generation is not deterministic, so replay several times and compare outcomes.
Last updated: October 2026
Production Observability, Tracing & Debugging
Exam Blueprint Focus: Operating Claude in enterprise production environments demands visibility far beyond traditional application performance monitoring (APM). In addition to standard server uptime and CPU metrics, developers must monitor semantic telemetry—including Time-to-First-Token (TTFT), Inter-Token Latency (ITL), prompt cache efficiency, and stop reason distributions. Candidates for the Anthropic Claude Certified Developer - Foundations (CCDV-F) exam must master distributed tracing using OpenTelemetry across multi-step autonomous agent lifecycles, diagnose complex runtime failures (such as infinite tool thrashing, schema parameter hallucination, and cache-invalidating dynamic timestamps), and implement replay debugging to isolate production anomalies.
Beyond Traditional Web Observability: The Generative AI Telemetry Stack
Traditional web microservices exhibit deterministic performance profiles: an HTTP request is received, a database query executes, and a response payload is returned. Observability in traditional systems centers on standard metrics: HTTP response status codes, server CPU utilization, memory consumption, and network I/O.
In generative AI systems, traditional metrics are fundamentally blind to application health:
An API call to Claude can return HTTP 200 OK while outputting a completely truncated JSON string, a hallucinated tool argument, or an apologetic refusal to answer.
Server CPU utilization may remain stable at 15% while user-perceived streaming latency degrades to unacceptable levels due to queuing or cache misses.
Token consumption may spike tenfold without any increase in user traffic due to multi-step agents entering circular tool-calling loops.
To maintain production reliability, architects must implement a dedicated Generative AI Telemetry Stack that captures operational performance across three distinct dimensions: Latency Dynamics , Token Economics & Caching Efficiency , and Semantic Reliability Signals . +-----------------------------------------------------------------------------------+
- | GENERATIVE AI TELEMETRY STACK |
- +-----------------------------------------------------------------------------------+
- | 1. Latency Dynamics |
- | - TTFT (Time-to-First-Token), ITL (Inter-Token Latency), End-to-End Duration |
- +-----------------------------------------------------------------------------------+
- | 2. Token Economics & Caching Telemetry |
- | - input_tokens, output_tokens, cache_creation_input_tokens, cache_read_tokens |
- | - Cache Hit Ratio: cache_read / (cache_creation + cache_read) |
- +-----------------------------------------------------------------------------------+
- | 3. Semantic Reliability & Quality Signals |
- | - Stop Reasons: end_turn, max_tokens, tool_use, stop_sequence, pause_turn, |
- | refusal |
- | - Tool error rates (is_error: true), client retries (429 rate limit, 529 ovld) |
- +-----------------------------------------------------------------------------------+
- Core LLM Telemetry Metrics
- 1. Latency Decomposition: TTFT, ITL, and End-to-End Duration
In streaming LLM architectures, total turnaround time is decomposed into distinct operational phases:
Time-to-First-Token (TTFT): The duration from when the client dispatches the request to when the first token chunk arrives via Server-Sent Events (SSE). TTFT captures:
Network transit to the Anthropic API endpoint.
Prompt compilation and prompt cache lookup.
The initial prefill phase (processing the input prompt tokens through the transformer network).
TTFT directly dictates perceived application responsiveness in interactive user interfaces. A low TTFT creates an immediate impression of speed, even if total response generation takes several seconds.
Inter-Token Latency (ITL): The time elapsed between consecutive token emissions during the generation phase ( content_block_delta events). ITL measures decode throughput, which varies by model and load. Variations in ITL produce visible streaming "jitter."
End-to-End (E2E) Latency: The total wall-clock duration from initial request dispatch until the final message_stop event is received and the connection closes. E2E latency is bounded by: Latency(E2E) = TTFT + (Output Tokens * ITL).
2. Token Consumption and Prompt Caching Economics
Every call to /v1/messages returns a detailed usage block. Production monitoring systems must extract, record, and aggregate these metrics across time: {
- "usage": {
- "input_tokens": 450,
- "output_tokens": 128,
- "cache_creation_input_tokens": 2840,
- "cache_read_input_tokens": 0
- }
- }
input_tokens : The count of raw, uncached prompt tokens processed during the call.
output_tokens : The count of tokens generated by Claude in the response.
cache_creation_input_tokens : The number of tokens written into Anthropic's ephemeral prompt cache (billed at 1.25x the base input rate for the 5-minute TTL, or 2x for the 1-hour TTL).
cache_read_input_tokens : The number of tokens retrieved from an existing warm cache prefix (billed at 0.1x the base input rate on most models, 0.05x on Claude Opus 5.5 and Claude Sonnet 5.5, and 0.025x on Claude Fable 5.1).
The Cache Hit Ratio Metric
Production observability dashboards must calculate and alert on the Prompt Cache Hit Ratio :
Cache Hit Ratio = cache_read_input_tokens / (cache_read_input_tokens + cache_creation_input_tokens)
Establish the normal hit ratio for each route and alert on sudden dips. A drop usually indicates cache prefix invalidation, which immediately inflates latency and multiplies the cost of the formerly cached input (10x at a 0.1x read multiplier, 20x at 0.05x).
3. Semantic Reliability Signals: Stop Reason Distributions
The stop_reason field returned in the Messages API response represents a vital operational health metric:
Stop Reason Meaning and Expected Behavior Alerting Threshold & Diagnostic Implication
end_turn The model naturally concluded its response or completed its conversational turn. Normal baseline for conversational completions; track its share per route.
tool_use The model halted text generation to invoke one or more tools defined in the request. Expected in autonomous agent loops. An unexpected spike may indicate tool thrashing.
max_tokens The model was abruptly cut off because output generation hit the configured max_tokens ceiling. CRITICAL ALERT: Indicates truncated responses, broken JSON syntax, or unclosed code blocks. Immediate investigation required!
stop_sequence Generation halted because the model emitted a configured custom stop sequence delimiter. Normal when using custom delimiters (e.g., </observation> or ===END=== ).
pause_turn A server-tool loop paused a long turn; send the returned content back to continue. Expected with server tools; a spike can mean long research turns.
refusal Claude (or a safety classifier) declined; stop_details carries the category. Returned with HTTP 200. Track by category; a spike after a prompt or model change needs review.
4. Tool Error Rates and Upstream API Health
In tool-calling architectures, telemetry must track:
Tool Error Rate: The percentage of tool_result blocks returning is_error: true . A high error rate indicates invalid parameter generation by Claude or downstream service failures.
Upstream Anthropic Status Codes: Monitoring HTTP 429 ( rate_limit_error ) and HTTP 529 ( overloaded_error ). Spikes in 429 require adjusting client-side token bucket algorithms, while 529 triggers exponential backoff and circuit breaker failover.
Distributed Tracing in Multi-Step Agent Architectures
Autonomous agents do not execute a single atomic LLM call. A user prompt initiates a complex, multi-step lifecycle involving intent classification, vector retrieval, iterative tool proposals, database queries, and final synthesis.
Without distributed tracing, diagnosing why an agent took 18 seconds to respond or why it failed a user request requires manually correlating disconnected logs across multiple services. [Trace ID: trace-agent-9b41a] Root Span: User Task Fulfillment (12.4s)
- │
- ├── Child Span 1: Intent Routing & Policy Check (Claude Haiku 5.5) [0.42s]
- │ └── Attributes: model=haiku, input_tokens=320, output_tokens=24
- │
- ├── Child Span 2: Vector Search & Knowledge Retrieval (Pinecone) [0.18s]
- │ └── Attributes: query_vector_dim=1536, top_k=5, score_threshold=0.82
- │
- ├── Child Span 3: Primary Agent Reasoning Step 1 (Claude Sonnet 5.5) [2.10s]
- │ └── Attributes: model=sonnet, stop_reason=tool_use, tool=query_sql_orders
- │
- ├── Child Span 4: Tool Execution: Production PostgreSQL DB [0.35s]
- │ └── Attributes: db.statement="SELECT * FROM orders...", is_error=false
- │
- ├── Child Span 5: Primary Agent Reasoning Step 2 (Claude Sonnet 5.5) [3.20s]
- │ └── Attributes: model=sonnet, stop_reason=tool_use, tool=initiate_refund
- │
- ├── Child Span 6: Tool Execution: Stripe Payment Gateway API [1.80s]
- │ └── Attributes: http.method=POST, http.status=200, refund_id=rf_99a
- │
- └── Child Span 7: Final Response Synthesis (Claude Sonnet 5.5) [4.35s]
- └── Attributes: model=sonnet, stop_reason=end_turn, output_tokens=340
- OpenTelemetry Semantic Conventions for Generative AI
To ensure interoperability across observability platforms (such as Datadog, Honeycomb, Dynatrace, and Jaeger), production implementations should adopt the OpenTelemetry Semantic Conventions for Generative AI systems:
- gen_ai.provider.name : "anthropic"
- gen_ai.request.model : "claude-sonnet-5-5"
- gen_ai.request.max_tokens : 4096
- gen_ai.usage.input_tokens : 1450 (Anthropic's input_tokens plus cache reads and writes)
- gen_ai.usage.output_tokens : 320
- gen_ai.usage.cache_read.input_tokens : 1200
- gen_ai.usage.cache_write.input_tokens : 0
- gen_ai.response.finish_reasons : ["end_turn"]
- gen_ai.response.time_to_first_chunk : TTFT for streamed responses
(Sampling attributes such as gen_ai.request.temperature exist in the conventions, but current Claude models reject non-default sampling values, so there is usually nothing to record.)
Diagnosing and Mitigating Critical Production Failure Modes
Operating Claude applications at scale exposes distinct failure modes that do not exist in conventional software stacks. Understanding these failure signatures is essential for both production reliability and the CCDV-F examination.
1. Infinite Agent Loops and Tool Thrashing
The Symptom: An autonomous agent consumes massive token budgets, runs for minutes, and eventually crashes with a timeout or maximum iteration error. Distributed traces reveal the model repeatedly toggling between two tools (e.g., checking order status, finding a missing parameter, querying customer records, and repeating).
Root Cause: Tool thrashing occurs when tool definitions are ambiguous or when a tool returns an uninformative error message (e.g., {"error": "Failed"} ). Unable to discern why the invocation failed, Claude re-attempts the same action or an opposing action without making progress.
Architectural Mitigations:
Hard Iteration Caps: Enforce a strict maximum iteration counter on the agent loop (e.g., max_turns = 10 ). If the cap is reached, break the loop immediately.
Informative Error Payloads: Tool handlers must return descriptive, actionable feedback inside tool_result blocks (e.g., {"is_error": true, "content": "Parameter 'customer_id' must start with 'CUST-'. Got '123'. Please correct the format."} ).
Loop Detection Heuristics: Track the rolling window of tool calls. If the identical tool name and parameter hash appears twice within 3 turns, inject a system intervention warning Claude of repetitive actions.
2. Hallucinated Tool Parameters Failing Schema Validation
The Symptom: Telemetry indicates elevated tool error rates ( is_error: true ), accompanied by client-side JSON Schema or Pydantic validation exceptions.
Root Cause: The model invents parameters that do not exist in the tool definition, outputs invalid enum values, or confuses parameter data types (e.g., passing a string "true" instead of a boolean true , or supplying an unformatted date string).
Architectural Mitigations:
Explicit Parameter Descriptions: Every parameter in input_schema must include clear descriptions, explicit enum constraints ( enum: ["active", "pending", "closed"] ), and regex validation patterns ( pattern: "^[0-9]{5}$" ).
Graceful Error Reflection: Never crash the server when parameter validation fails. Catch the Pydantic ValidationError and feed the structured validation message back into the conversation as a tool_result with is_error: true . Claude will read the validation error and self-correct on its next turn.
3. Unexpected Truncation via max_tokens Ceilings
The Symptom: Downstream JSON parsers throw syntax errors ( json.decoder.JSONDecodeError: Unterminated string ), markdown tables are missing closing delimiters, or generated code stops mid-function.
Root Cause: The generation exceeded the configured max_tokens limit. With thinking , which is adaptive and on by default on Claude Sonnet 5.5 and the rest of the current lineup, developers must remember that thinking tokens count toward the overall max_tokens budget. If max_tokens is set to 2,048 and thinking consumes 1,500 tokens, only 548 tokens remain for the visible output payload!
Architectural Mitigations:
Stop Reason Monitoring: Always inspect response.stop_reason . If stop_reason == "max_tokens" , treat the response as potentially corrupt.
Budget Sizing: Size max_tokens generously (e.g., 4,096 or 8,192). In Claude's pricing model, max_tokens is a ceiling, not a pre-allocation fee; you are billed strictly for tokens actually generated.
Continuation Looping: For long-form generation, detect stop_reason == "max_tokens" and send a follow-up user message that quotes the end of the partial output and asks Claude to continue from there (a trailing assistant prefill is rejected on Claude 4.6 and later models).
4. Cache Misses Caused by Dynamic Prefix Timestamps
The Symptom: Following a prompt update, API billing surges and TTFT latency jumps. Telemetry shows cache_read_input_tokens = 0 across all requests.
Root Cause: Anthropic prompt caching relies on exact, deterministic prefix matching. If a developer dynamically injects varying data at the start of the system prompt (e.g., f"Current system time: {datetime.now().isoformat()}\n\n{STATIC_POLICY_DOCS}" ), the initial token sequence changes on every single request. This invalidates the cache prefix, preventing any cache hits.
Architectural Mitigations:
Static Prefix Ordering: Place all static, heavy content (system instructions, tool definitions, reference manuals) at the very beginning of the context window with the cache_control: {"type": "ephemeral"} breakpoint.
Append Dynamic Variables Last: Inject dynamic, per-request data (current timestamp, user session IDs, ephemeral query parameters) at the end of the system prompt or inside the final user message, strictly after the cached prefix.
Replay Debugging: Incident Reproduction
When an agent misbehaves in production, debugging the incident locally is notoriously challenging if the runtime environment state is lost. Replay debugging is the architectural practice of capturing the complete, exact request payload and intermediate tool states, enabling engineers to replay the transaction in local development environments. Tool results can be replayed exactly from the recording; the model's own output is not deterministic, so run the replay several times and compare.
- The Replay Capture Architecture [Production Incident Occurs] (e.g., Tool Loop or Exception)
- │
- ▼
- +─────────────────────────────────────────────────────────────+
- | PRODUCTION TELEMETRY RECORDER |
- | 1. Capture system prompt, model ID & request params (effort)|
- | 2. Serialize exact messages array (all turns & tool calls) |
- | 3. Mask PII / credentials via automated regex redactor |
- | 4. Record intermediate mock tool_result payloads |
- | 5. Persist replay artifact to S3 / Debug Store |
- +─────────────────────────────────────────────────────────────+
- │
- ▼
- [replay_session_INC_402.json]
- │
- ▼
- +─────────────────────────────────────────────────────────────+
- | LOCAL REPLAY HARNESS (`pytest --replay`) |
- | 1. Loads sanitized messages array into local client |
- | 2. Mocks tool responses using recorded payloads |
- | 3. Allows engineer to inspect step-by-step reasoning |
- | 4. Tests prompt & schema adjustments over repeated runs |
- +─────────────────────────────────────────────────────────────+
- Sanitization and Security in Replay Capture
Because replayed payloads contain full conversational histories, production recording pipelines must enforce strict data governance:
PII Scrubbing: Pass all serialized message arrays through deterministic regex scrubbers to mask credit card numbers, social security numbers, and email addresses.
Credential Masking: Strip API keys, session tokens, and bearer credentials from tool arguments before saving replay artifacts.
Ephemeral Retention: Configure strict TTL policies (e.g., 7-day automatic deletion) on debug replay storage buckets to comply with GDPR and enterprise data retention policies.
Comparative Matrix: Production Failure Modes, Telemetry Signatures, and Architectural Mitigations
The following matrix provides a diagnostic reference for identifying and remediating runtime anomalies in Claude applications:
Failure Mode Telemetry Signature Primary Metric Indicator Root Cause Architectural Mitigation
Infinite Agent Loop / Tool Thrashing High end-to-end duration, multi-step trace with alternating tool spans Runaway input_tokens and output_tokens per trace; turn count > 8 Ambiguous tool schemas; uninformative is_error feedback Enforce max_turns limit; return actionable error descriptions; implement loop detection heuristics.
Tool Parameter Hallucination Repeated tool spans followed immediately by is_error: true results High tool error rate ( is_error: true > 5%) Underspecified schema; missing enum constraints or regex Enforce strict Pydantic schemas; return descriptive validation errors back to Claude; add enum constraints.
Unexpected Response Truncation Prematurely terminated outputs; JSON syntax parsing exceptions Response stop_reason == "max_tokens" Low max_tokens setting; thinking tokens exhausting the shared ceiling Increase max_tokens ; inspect stop_reason programmatically; implement continuation request loop.
Dynamic Cache Invalidation Sudden surge in input token billing; elevated Time-to-First-Token cache_read_input_tokens == 0 ; cache_creation_input_tokens high Dynamic timestamps or UUIDs placed at the start of system prompt Reorder prompt layout: static prefix first with cache_control , dynamic variables appended at end.
Production Implementation: OpenTelemetry Instrumentation & Replay Capture
The following complete Python implementation illustrates how to instrument Claude API requests and tool invocations using the OpenTelemetry SDK. It captures latency phases, logs standard GenAI semantic attributes, monitors stop reasons, and serializes replay payloads when anomalies occur: import json
- import time
- from typing import Dict, Any, List, Optional
- from opentelemetry import trace
- from opentelemetry.trace import Status, StatusCode
- import anthropic
- # Initialize OpenTelemetry Tracer
- tracer = trace.get_tracer("claude-production-observability", "1.0.0")
- client = anthropic.Anthropic()
- def execute_instrumented_agent_turn(
- session_id: str,
- system_prompt: str,
- messages: List[Dict[str, Any]],
- tools: Optional[List[Dict[str, Any]]] = None,
- max_tokens: int = 2048
) -> Dict[str, Any]:
"""
Executes a monitored Claude invocation within an OpenTelemetry span.
Captures GenAI semantic conventions, tracks TTFT/E2E latency, and logs stop reasons.
"""
with tracer.start_as_current_span("claude_messages_invocation") as span:
- # 1. Set OpenTelemetry GenAI Semantic Convention Attributes
- span.set_attribute("gen_ai.provider.name", "anthropic")
- span.set_attribute("gen_ai.request.model", "claude-sonnet-5-5")
- span.set_attribute("gen_ai.request.max_tokens", max_tokens)
- span.set_attribute("session.id", session_id)
start_time = time.time()
try:
- # 2. Dispatch streaming request to measure TTFT and E2E Latency
- first_token_time: Optional[float] = None
- accumulated_text = ""
- with client.messages.stream(
- model="claude-sonnet-5-5",
- max_tokens=max_tokens,
- system=system_prompt,
- messages=messages,
- tools=tools or []
) as stream:
for text_delta in stream.text_stream:
if first_token_time is None:
- first_token_time = time.time()
- # OTel convention: time to first chunk, in seconds
- span.set_attribute("gen_ai.response.time_to_first_chunk", first_token_time - start_time)
- accumulated_text += text_delta
- # Obtain the final aggregated message object
- final_response = stream.get_final_message()
- end_time = time.time()
- e2e_duration_ms = (end_time - start_time) * 1000
- span.set_attribute("app.latency.e2e_ms", e2e_duration_ms) # app key, not a gen_ai convention
- # 3. Record Token Usage and Cache Telemetry
- usage = final_response.usage
- cache_creation = getattr(usage, "cache_creation_input_tokens", 0) or 0
- cache_read = getattr(usage, "cache_read_input_tokens", 0) or 0
# Anthropic's input_tokens excludes cached tokens; the OTel value includes them.
- span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens + cache_read + cache_creation)
- span.set_attribute("gen_ai.usage.output_tokens", usage.output_tokens)
- span.set_attribute("gen_ai.usage.cache_write.input_tokens", cache_creation)
- span.set_attribute("gen_ai.usage.cache_read.input_tokens", cache_read)
- # 4. Monitor Stop Reason and Detect Anomalies
- stop_reason = final_response.stop_reason
- span.set_attribute("gen_ai.response.finish_reasons", [stop_reason])
if stop_reason == "max_tokens":
- span.set_status(Status(StatusCode.ERROR, "Generation truncated due to max_tokens ceiling"))
- save_replay_debug_artifact(session_id, system_prompt, messages, final_response)
else:
span.set_status(Status(StatusCode.OK))
- return {
- "response": final_response,
- "ttft_ms": (first_token_time - start_time) * 1000 if first_token_time else None,
- "e2e_ms": e2e_duration_ms
- }
except Exception as exc:
- span.record_exception(exc)
- span.set_status(Status(StatusCode.ERROR, str(exc)))
- save_replay_debug_artifact(session_id, system_prompt, messages, error=str(exc))
- raise exc
- def save_replay_debug_artifact(
- session_id: str,
- system_prompt: str,
- messages: List[Dict[str, Any]],
- response: Any = None,
- error: Optional[str] = None
) -> None:
"""
Serializes runtime payload into a replay artifact for local debugging.
- """
- replay_payload = {
- "session_id": session_id,
- "timestamp": time.time(),
- "error": error,
- "stop_reason": getattr(response, "stop_reason", None) if response else None,
- "request": {
- "system_prompt": system_prompt,
- "messages": messages,
- }
- }
- print(f"[REPLAY DEBUG CAPTURED] Session: {session_id} - Artifact serialized for debugging.")
- Common Operational Anti-Patterns
Ignoring the stop_reason Field: Treating all completed HTTP requests as successful responses without inspecting whether stop_reason is max_tokens . This leads to silent downstream data corruption and parsing crashes.
Prefix Invalidation via Dynamic Timestamps: Inserting current dates or millisecond timestamps at line 1 of the system prompt, causing a complete failure of Anthropic prompt caching across 100% of user requests.
Unbounded Multi-Step Agent Execution: Allowing an autonomous agent loop to run without a strict max_turns iteration ceiling, creating financial risk from runaway tool-thrashing loops.
Discarding Failed Interaction Traces: Failing to persist the complete conversation history and tool outputs when an agent crashes, forcing developers to rely on guesswork rather than replay debugging.
Evaluating Latency Solely by End-to-End Duration: Neglecting to separate Time-to-First-Token (TTFT) from Inter-Token Latency (ITL). A high E2E latency on a 3,000-token response is expected, but a high TTFT on a simple request indicates prompt compilation bottlenecks or cache misses.