Study guide
10.3 Batch Inference & Bedrock Intelligent Prompt Routing
AWS Certified Generative AI Developer - Professional AIP-C01
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & Foundation Model Fundamentals
1.1 AIP-C01 Exam Structure, Logistics & Passing Strategy 1.2 AIP-C01 Scoring & Professional Decision Strategy 1.3 Foundation Model Landscape & Model Selection on Amazon Bedrock 1.4 Model Abstraction, Controlled Switching & Resilience 1.5 Customization Spectrum: Prompting vs RAG vs Fine-Tuning vs Continued Pre-training 1.6 Custom Model Deployment & Lifecycle Management
2 Chapter 2: Bedrock Knowledge Bases & RAG Architectures
2.1 Managed RAG Architecture with Bedrock Knowledge Bases 2.2 Knowledge Source Integration, Synchronization & Freshness 2.3 Data Ingestion, Parsing & Chunking Strategies 2.4 Embedding Models & Vector Representations 2.5 Vector Index Scale, Migration & Retrieval Evaluation 2.6 Vector Database Selection: OpenSearch Serverless vs Aurora pgvector vs Managed Stores 2.7 Vector Store Security, Multi-Tenancy & Connectivity
3 Chapter 3: Advanced Vector Search, Indexing & Data Compliance
3.1 Hybrid Search, Metadata Filtering & Semantic Reranking 3.2 Metadata Filtering, Reranking & Query Transformation 3.3 Fine-Tuning & Continued Pre-Training Data Pipeline Engineering 3.4 Data Validation, Multimodal Processing & Model Formatting 3.5 Data Privacy, Data Residency & Compliance Governance 3.6 Privacy Preservation, Source Lineage & Audit Evidence
4 Chapter 4: Prompt Engineering & Bedrock Runtime APIs
4.1 Advanced Prompt Engineering Techniques & Inference Parameters 4.2 Inference Parameters, Prompt Governance & Regression 4.3 Converse API vs InvokeModel API & Message Formatting 4.4 Tool Calling, FM API Resilience & Model Routing 4.5 Streaming Architectures & Asynchronous Event Processing 4.6 Resilient Streaming, Deployment & Asynchronous Processing
5 Chapter 5: Amazon Bedrock Agents & Tool Integration
5.1 Agentic Architecture, ReAct Reasoning Loops & Orchestration 5.2 Agent Frameworks, MCP & Lifecycle Management 5.3 Action Groups, OpenAPI Schemas & AWS Lambda Integration 5.4 Agent Memory, Session State & Multi-Agent Collaboration Patterns 5.5 Persistent State & Multi-Agent Coordination 5.6 Advanced Agent Prompt Templates & Flow Customization 5.7 Agent Safeguards, Tool Reliability & Cost Control
6 Chapter 6: Workflow Orchestration & Application Integration
6.1 Amazon Bedrock Prompt Flows & Visual Orchestration 6.2 Step Functions Orchestration & Human-in-the-Loop Workflows 6.3 Enterprise Integration: API Gateway, AppSync, and EventBridge 6.4 Hybrid Integration, GenAI Gateways & CI/CD Tooling
7 Chapter 7: AI Safety & Guardrails for Amazon Bedrock
7.1 Guardrails Architecture, Denied Topics & Content Filters 7.2 Content Safety, Defense in Depth & Threat Detection 7.3 Sensitive Information Filters & PII Redaction/Masking 7.4 Contextual Grounding Checks & Hallucination Prevention
8 Chapter 8: Security, Access Control & Network Architecture
8.1 IAM Least Privilege & Resource Policies for Bedrock 8.2 PrivateLink VPC Endpoints & Network Isolation 8.3 Encryption with AWS KMS & Cross-Account Access Patterns
9 Chapter 9: Governance, Auditing & Operational Logging
9.1 SageMaker Model Cards & AI Governance Frameworks 9.2 Lineage, Fairness, Transparency & Continuous Governance 9.3 Invocation Logging, CloudTrail Auditing & Observability 9.4 Responsible AI, Bias Mitigation & Red Teaming Methodologies
10 Chapter 10: Performance, Throughput & Cost Optimization
10.1 On-Demand vs Provisioned Throughput & Capacity Planning 10.2 Capacity Measurement, Scaling Patterns & Resilience 10.3 Batch Inference & Bedrock Intelligent Prompt Routing 10.4 Intelligent Prompt Routing & Application Eligibility 10.5 Prompt Caching, Token Optimization & Latency Reduction
11 Chapter 11: Testing, Evaluation & Troubleshooting GenAI Solutions
11.1 Amazon Bedrock Model Evaluation (Automatic & Human Workflows) 11.2 Human, RAG & Continuous Evaluation 11.3 Evaluation Metrics & RAG Quality Benchmarking (Ragas, ROUGE, BLEU) 11.4 RAG and Agent Performance Evaluation 11.5 CloudWatch Metrics, Throttling & Operational Observability 11.6 Quota Diagnosis, Retry Control & Operational Troubleshooting 11.7 Troubleshooting Common GenAI Failure Modes & Agent Traces 11.8 Prompt Maintenance, Content Handling & Deployment Validation
10.3 Batch Inference & Bedrock Intelligent Prompt Routing
Key Takeaways Explain This Topic AI
Batch output ordering is not guaranteed, so correlate input and output with record IDs.
Configured prompt routers choose exactly two supported models in the same family and use a fallback plus quality-difference criterion.
Application routing remains necessary when policy, modality, residency, or tool support determines model eligibility.
Last updated: September 2026
10.3 Batch Inference & Bedrock Intelligent Prompt Routing
Batch inference and intelligent prompt routing solve different optimization problems. Batch moves noninteractive work into an asynchronous job. Prompt routing chooses between supported models for each interactive request. Neither feature removes the need to validate model support, IAM, quotas, output quality, or cost.
Batch inference workflow
Use batch inference for large offline workloads such as nightly classification, document summarization, synthetic-data generation, or evaluation preparation. The application writes request records to Amazon S3, submits a model invocation job with a service role, monitors job status, and reads output JSON Lines files from S3.
Each input line contains a modelInput object whose schema matches the selected invocation type. A recordId is useful for correlation; current Bedrock documentation says that if it is omitted, Bedrock adds one to the output. Output order is not guaranteed to match input order, so join by record ID rather than array position. If records reference S3 media, follow the model and input-folder rules exactly.
- Batch concern Correct design response
- Large input set Split within current file, record, and job quotas; validate every JSONL line before submission
- Sensitive S3 data Restrict the job role to exact input/output prefixes and required KMS keys
- Partial failures Parse per-record results and retry only eligible failed records with idempotent IDs
Cost Confirm that the selected model receives current batch pricing; AWS advertises lower batch pricing for select models
Completion time Treat batch as asynchronous; do not use it for a user waiting on a response
Batch inference has its own service quotas and supported-model table. This separation makes it suitable for offline load, but avoid claiming that a batch job can never affect any other workload. Design each path to its documented quotas and observe both.
Job lifecycle and error recovery
Validate the entire input folder before submission. Check JSONL syntax, unique record IDs, modelInput schema, referenced media paths, file sizes, record counts, and encryption access. The submitting identity needs permission to create and inspect jobs; the service role needs the exact S3 and KMS access required to perform them. These are different trust relationships.
Write outputs to a job-specific prefix. When the job completes, parse success and error records, verify expected record counts, and create a retry file containing only eligible failures. Preserve the original record ID and add an attempt identifier outside the model payload so downstream processing remains idempotent. Do not resubmit successful state-changing work.
Monitor queue time, run time, success rate, failure categories, tokens, output size, and cost. Set an operational deadline after which the workflow alerts or falls back; asynchronous does not mean unbounded. Lifecycle rules can archive or delete input and output only after audit and consumer requirements are satisfied.
Batch inference uses a selected model for the job. It is not intelligent prompt routing over each line. If records need different models, partition them through application logic or separate jobs and retain correlation across the workflow.
Data quality gates for batch
Run a preflight sample through the exact model and parser before submitting the full corpus. Validate not only job acceptance but downstream output schema, language, safety, and token size. A syntactically successful job can generate unusable business records at scale.
Partition sensitive and nonsensitive records when they require different roles, keys, retention, or output destinations. Keep each job's manifest and configuration so a result can be reproduced. If input data changes during a long job, decide whether the output represents the submission snapshot or must be invalidated.
Use a completion event or controlled poller to start downstream processing once. Duplicate job-completion notifications must be harmless. Verify counts and checksums before marking the business batch complete, and surface a partial-completion state rather than silently dropping failed records.
Batch security boundary
Block public access on input and output buckets, require transport encryption, use customer-managed keys when policy calls for them, and restrict the Bedrock service role with source-account and source-ARN conditions where supported. Review generated outputs for sensitive material before distributing them to a wider analytics bucket.
Keep model access explicit in the job role and review it when the batch model changes. A broad wildcard permission can make a future, more expensive or less appropriate model invokable without the evaluation and budget approval that the workflow requires.