Study guide

10.2 Capacity Measurement, Scaling Patterns & Resilience

AWS Certified Generative AI Developer - Professional AIP-C01

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Exam Overview & Foundation Model Fundamentals

1.1 AIP-C01 Exam Structure, Logistics & Passing Strategy 1.2 AIP-C01 Scoring & Professional Decision Strategy 1.3 Foundation Model Landscape & Model Selection on Amazon Bedrock 1.4 Model Abstraction, Controlled Switching & Resilience 1.5 Customization Spectrum: Prompting vs RAG vs Fine-Tuning vs Continued Pre-training 1.6 Custom Model Deployment & Lifecycle Management

2 Chapter 2: Bedrock Knowledge Bases & RAG Architectures

2.1 Managed RAG Architecture with Bedrock Knowledge Bases 2.2 Knowledge Source Integration, Synchronization & Freshness 2.3 Data Ingestion, Parsing & Chunking Strategies 2.4 Embedding Models & Vector Representations 2.5 Vector Index Scale, Migration & Retrieval Evaluation 2.6 Vector Database Selection: OpenSearch Serverless vs Aurora pgvector vs Managed Stores 2.7 Vector Store Security, Multi-Tenancy & Connectivity

3 Chapter 3: Advanced Vector Search, Indexing & Data Compliance

3.1 Hybrid Search, Metadata Filtering & Semantic Reranking 3.2 Metadata Filtering, Reranking & Query Transformation 3.3 Fine-Tuning & Continued Pre-Training Data Pipeline Engineering 3.4 Data Validation, Multimodal Processing & Model Formatting 3.5 Data Privacy, Data Residency & Compliance Governance 3.6 Privacy Preservation, Source Lineage & Audit Evidence

4 Chapter 4: Prompt Engineering & Bedrock Runtime APIs

4.1 Advanced Prompt Engineering Techniques & Inference Parameters 4.2 Inference Parameters, Prompt Governance & Regression 4.3 Converse API vs InvokeModel API & Message Formatting 4.4 Tool Calling, FM API Resilience & Model Routing 4.5 Streaming Architectures & Asynchronous Event Processing 4.6 Resilient Streaming, Deployment & Asynchronous Processing

5 Chapter 5: Amazon Bedrock Agents & Tool Integration

5.1 Agentic Architecture, ReAct Reasoning Loops & Orchestration 5.2 Agent Frameworks, MCP & Lifecycle Management 5.3 Action Groups, OpenAPI Schemas & AWS Lambda Integration 5.4 Agent Memory, Session State & Multi-Agent Collaboration Patterns 5.5 Persistent State & Multi-Agent Coordination 5.6 Advanced Agent Prompt Templates & Flow Customization 5.7 Agent Safeguards, Tool Reliability & Cost Control

6 Chapter 6: Workflow Orchestration & Application Integration

6.1 Amazon Bedrock Prompt Flows & Visual Orchestration 6.2 Step Functions Orchestration & Human-in-the-Loop Workflows 6.3 Enterprise Integration: API Gateway, AppSync, and EventBridge 6.4 Hybrid Integration, GenAI Gateways & CI/CD Tooling

7 Chapter 7: AI Safety & Guardrails for Amazon Bedrock

7.1 Guardrails Architecture, Denied Topics & Content Filters 7.2 Content Safety, Defense in Depth & Threat Detection 7.3 Sensitive Information Filters & PII Redaction/Masking 7.4 Contextual Grounding Checks & Hallucination Prevention

8 Chapter 8: Security, Access Control & Network Architecture

8.1 IAM Least Privilege & Resource Policies for Bedrock 8.2 PrivateLink VPC Endpoints & Network Isolation 8.3 Encryption with AWS KMS & Cross-Account Access Patterns

9 Chapter 9: Governance, Auditing & Operational Logging

9.1 SageMaker Model Cards & AI Governance Frameworks 9.2 Lineage, Fairness, Transparency & Continuous Governance 9.3 Invocation Logging, CloudTrail Auditing & Observability 9.4 Responsible AI, Bias Mitigation & Red Teaming Methodologies

10 Chapter 10: Performance, Throughput & Cost Optimization

10.1 On-Demand vs Provisioned Throughput & Capacity Planning 10.2 Capacity Measurement, Scaling Patterns & Resilience 10.3 Batch Inference & Bedrock Intelligent Prompt Routing 10.4 Intelligent Prompt Routing & Application Eligibility 10.5 Prompt Caching, Token Optimization & Latency Reduction

11 Chapter 11: Testing, Evaluation & Troubleshooting GenAI Solutions

11.1 Amazon Bedrock Model Evaluation (Automatic & Human Workflows) 11.2 Human, RAG & Continuous Evaluation 11.3 Evaluation Metrics & RAG Quality Benchmarking (Ragas, ROUGE, BLEU) 11.4 RAG and Agent Performance Evaluation 11.5 CloudWatch Metrics, Throttling & Operational Observability 11.6 Quota Diagnosis, Retry Control & Operational Troubleshooting 11.7 Troubleshooting Common GenAI Failure Modes & Agent Traces 11.8 Prompt Maintenance, Content Handling & Deployment Validation

10.2 Capacity Measurement, Scaling Patterns & Resilience

Key Takeaways Explain This Topic AI

Measure both input and output throughput and use the binding capacity requirement.

Do not claim instant Application Auto Scaling support for Provisioned Throughput without current documentation.

Bound concurrency, ramp load gradually, and define queue, fallback, timeout, and retry policies.

Last updated: September 2026

  • 10.2 Capacity Measurement, Scaling Patterns & Resilience
  • Measure the workload before purchasing capacity

Collect input tokens, requested and actual output tokens, request rate, concurrency, latency percentiles, streaming duration, retry volume, and traffic seasonality. Separate interactive traffic from offline processing. Measure the full application path because retrieval, Guardrails, tools, and network hops contribute to user latency even when model inference is stable.

Use controlled load tests with representative prompt sizes. Ramp traffic gradually and watch successful calls, throttles, transient capacity errors, queue depth, and latency. A test made only of tiny prompts can conceal the token pressure of production RAG requests.

If a hypothetical model unit supports 300,000 input tokens per minute and 60,000 output tokens per minute, and the measured peak requires 600,000 input tokens and 150,000 output tokens per minute, calculate both constraints:

Input requirement: ceiling of 600,000 divided by 300,000 equals 2 units.

Output requirement: ceiling of 150,000 divided by 60,000 equals 3 units.

Required capacity is the larger result, so the answer is 3 units.

This arithmetic is useful only when the request rate and unit specification are valid. Concurrency multiplied by 60 divided by latency is an estimate of completed requests per minute under a steady pipeline, not a universal capacity formula. Validate it with a load test and headroom for variance.

Scaling patterns

Do not claim that Amazon Bedrock Provisioned Throughput is a scalable target in Application Auto Scaling unless current service documentation explicitly supports that integration. The Bedrock control plane can create, inspect, modify, and delete supported throughput resources, but provisioning is not an instant per-request autoscaling mechanism. Capacity changes require quotas, lead time, supported settings, and deployment controls.

For predictable cycles, schedule an operational review and modify capacity ahead of the event when supported. For unexpected bursts, bound client concurrency, queue noninteractive work, use cross-Region inference when policy allows, apply backoff with jitter, and degrade gracefully. A fallback model is acceptable only if it still meets the task's safety and quality contract.

Route offline summarization, embedding, or bulk generation to batch processing when supported. This keeps interactive paths focused on user-facing latency and uses the separate batch workflow. Semantic or prompt caching can avoid duplicate computation, but cache hits are workload- and model-dependent.

Cost and reliability decision

Estimate monthly cost under realistic utilization, including cache reads and writes, retries, evaluation traffic, vector search, logging, and idle provisioned hours. Compare cost per successful business task. A fixed allocation can be economical for sustained traffic but wasteful for a nightly ten-minute job. On-demand can be economical for irregular traffic but needs quota and capacity handling.

Define service objectives and failure policies: maximum queue delay, acceptable fallback, timeout, retry budget, and what happens when all model paths are unavailable. Monitor the exact endpoint and target used. Current AWS guidance distinguishes bedrock-runtime and bedrock-mantle quota accounting; some models expose requests-per-minute quotas, while token accounting and burndown behavior vary. Check the quotas for the exact model rather than teaching a universal RPM-plus-TPM rule.

The professional-level answer combines measurement, current support checks, economics, and resilience. Provisioned Throughput is not synonymous with unlimited or error-free capacity, and custom model is no longer synonymous with mandatory Provisioned Throughput.

Load-test design and headroom

Replay representative short, median, and long prompts with realistic output limits and concurrency. Increase traffic in controlled steps, hold each level long enough to observe queues and tail latency, and capture request IDs for failures. Test the steady baseline, expected peak, sudden burst, dependency slowdown, and Region or model failure path.

Headroom covers variance and recovery, but it should be explicit. Document the maximum accepted concurrency, queue depth, queue-age objective, retry budget, and threshold for degrading to retrieval-only or delayed processing. If a fallback model produces a different schema or weaker safety behavior, it is not compatible until the application adapts and evaluates it.

Capacity changes are deployments. Verify quotas and supported unit changes, schedule them before known events, observe the new allocation, and maintain rollback. Do not make a long commitment automatically in reaction to a brief alarm. Cost and reliability approvals should reflect the commitment term.

Recovery objective

State how long the service may be degraded and how much queued work may be lost or replayed. These objectives determine whether an explicit failure, a delayed batch result, or a compatible fallback is acceptable. Test recovery, not only failover.