Study guide

High Availability, Failover, Chaos, and Self-Healing

AWS Certified Solutions Architect - Professional SAP-C02

Track your progress as you read

Finished sections get a check mark.

1 Introduction and SAP-C02 Exam Facts

Current SAP-C02 Exam Facts Exam Format, Scoring, and Retakes Candidate Profile and Four Exam Domains

2 Multi-Account Governance

Multi-Account Strategy, Account Topologies, and OU Hierarchies Service Control Policies, Inheritance, and Permission Guardrails Control Tower, Landing Zones, Central Logging, and Resource Sharing

3 Hybrid and Inter-VPC Connectivity

Transit Gateway, VPC Connectivity, and PrivateLink Direct Connect, VPN, and Hybrid Redundancy Hybrid DNS with Route 53 Resolver Segmentation, Traffic Monitoring, and Service Endpoints

4 Identity Federation and Cross-Account Access

IAM Identity Center, External IdPs, and Workforce Federation Cross-Account Roles, STS, and Resource Policies Permissions Boundaries, Delegated Administration, and Least Privilege at Scale

5 Data Protection, Encryption, and Compliance

KMS, Multi-Region Keys, Envelope Encryption, and Certificates Encryption in Transit, Security Groups, NACLs, and VPC Endpoints Centralized Audit, Security Hub, GuardDuty, Inspector, and Access Analyzer

6 Deployment, Compute, Containers, and Serverless

Infrastructure as Code, CI/CD, Change Management, and Rollback EC2 Families, Auto Scaling, and Managed Compute ECS, EKS, Fargate, and ECR at Enterprise Scale Lambda, Step Functions, API Gateway, and Configuration Management

7 Storage, Caching, and Edge Performance

S3 Storage Classes, Object Lock, Replication, and Access Paths EBS, EFS, FSx Family, and Storage Gateway CloudFront Behaviors, Origin Access, Edge Compute, and Global Accelerator ElastiCache, MemoryDB, DAX, and Buffering versus Cache versus Replicas

8 Purpose-Built and Global Databases

Aurora Storage, Replicas, and Global Database DynamoDB Keys, Global Tables, and DAX Purpose-Built Database Selection

9 Integration, Messaging, and Streaming

API Gateway Architectures Decoupled Messaging with Amazon SQS and Amazon SNS Event-Driven Design with Amazon EventBridge Streaming Analytics with Amazon Kinesis and Amazon MSK

10 Resilience and Disaster Recovery

Disaster Recovery Patterns Route 53 Routing and Application Recovery Controller Backup, Replication, and Service Quotas High Availability, Failover, Chaos, and Self-Healing

11 Operational Excellence and Automation

CloudWatch Centralized Telemetry Distributed Tracing with AWS X-Ray Systems Manager Automation Deployment Strategies and Auto-Remediation

12 Cost Optimization and Visibility

Cost Explorer, Budgets, CUR, and Allocation Tags Savings Plans, Reserved Instances, Coverage, and Sharing Spot, Mixed Auto Scaling, Compute Optimizer, and Storage Lens Data Transfer Modeling: NAT, Endpoints, CloudFront, and Hybrid

13 Improve Security and Performance

Secrets, Least Privilege Audits, and Traceability Config Rules, Patching, Backup, and Regulatory Controls Performance Tuning for Existing Workloads

14 Migration Assessment and Wave Planning

Seven Migration Strategies (7 Rs) and Workload Selection Application Discovery, Migration Hub, and Migration Evaluator Wave Planning, Prioritization, and Assessment Governance

15 Migration Tools and Data Transfer

AWS Application Migration Service (MGN) AWS DMS, SCT, and Change Data Capture Large-Scale Data Transfer and Landing-Zone Prerequisites

16 Target Architecture, Modernization, and Cutover

Selecting Compute, Storage, and Database Platforms for Migrated Workloads Container Modernization: ECS, EKS, Fargate, ECR, and Strangler Fig Serverless, Decoupling, Purpose-Built Databases, and Integration Services Cutover, Rollback, and Unscored Emerging AI Topics

High Availability, Failover, Chaos, and Self-Healing

Key Takeaways Explain This Topic AI

EC2 simplified automatic recovery (default on supported instances) and CloudWatch action-based recovery respond to failed system status checks by migrating the instance; RAM is lost. They do not replace Multi-AZ load balancing.

Loosely coupled failovers use Amazon SQS, Amazon SNS, and AWS Step Functions so a downstream outage becomes backlog and retries instead of a synchronous outage of the caller.

AWS Fault Injection Service (AWS FIS) is current: experiment templates, actions, targets, stop conditions, multi-account experiments, and a Scenario Library that includes AZ Availability: Power Interruption. FIS performs real actions on real resources.

Self-healing (Auto Scaling, alarms that restart, zonal autoshift) still needs runbooks for data corruption, split-brain, quota denial, and human approval gates.

Task 3.4 tests existing stacks with game days. Task 2.4 designs decoupling into new solutions. Task 1.3 standardizes organizational experiment guardrails and delegated incident roles.

Last updated: September 2026

Fail small, fail tested, fail without a conference call

Quick Answer: Keep individual EC2 instances recoverable with simplified automatic recovery or CloudWatch action-based recovery . Keep applications recoverable with Multi-AZ , decoupled APIs ( Amazon Simple Queue Service (Amazon SQS) , Amazon Simple Notification Service (Amazon SNS) , AWS Step Functions ), and traffic shift (load balancers, zonal shift). Prove it with disaster recovery (DR) drills and AWS Fault Injection Service (AWS FIS) . Automate what is safe; keep a runbook for what is not.

High availability ( HA ) is “survive the failure of a component without a Regional declaration of disaster.” Disaster recovery is “the Region, the account, or the data is gone.” SAP-C02 fails candidates who answer every stem with Aurora Global Database when the stem is a system status check on one instance, and who answer every stem with auto recovery when the stem is a poison message taking down a tightly coupled payments API.

Auto recovery is not architecture

Amazon EC2 system status checks watch the underlying host (network, power, hypervisor software). Instance status checks watch the guest (kernel hangs, exhausted network in the OS). Automatic instance recovery runs only when a system check fails, and only if you configured a mechanism before the failure.

Mechanism Configuration What is preserved What is lost Extra cost

Simplified automatic recovery Default on for supported instance types Instance ID, public/private/Elastic IPs, metadata, placement group, attached EBS volumes, AZ RAM ; OS uptime resets. Not for instances that attach instance store at launch None beyond the instance

CloudWatch action-based recovery You create the alarm and recover action Same identity and networking as above RAM and instance-store volume data CloudWatch charges

Recovery looks like an unplanned reboot on a new host. If recovery fails, AWS may leave the instance on the bad host; the documented manual step is stop and start (new public IPv4 unless you use an Elastic IP). Metal sizes can use CloudWatch recovery where simplified recovery is excluded. Neither mechanism moves the instance to another AZ or Region. AWS’s own guidance after describing auto recovery is to put instances behind Elastic Load Balancing and Amazon EC2 Auto Scaling so traffic fails over to healthy instances instead of waiting on a single guest.

For existing fleets (Task 3.4), turning on CloudWatch recovery is a cheap win for dedicated instances that cannot yet be stateless. For new solutions (Task 2.4), do not design a singleton EC2 with recovery as the HA story.

Loosely coupled failovers

A tightly coupled payments API that HTTP POST s to a ledger in the same AZ will time out when that AZ is impaired. The Well-Architected reliability skill is to fail over to healthy resources without making the caller wait on the impaired path.

Amazon SQS buffers work. Standard queues maximize throughput and retries; FIFO queues preserve order and exactly-once processing within the window SQS documents. A dead-letter queue (DLQ) captures messages that exceed maxReceiveCount so poison payloads do not infinite-loop. The API’s RTO becomes “accept the message,” not “commit the ledger in-line.”

Amazon SNS fans out the same event to SQS queues, Lambda, HTTPS, and more. That is how you notify fraud, analytics, and customer-email without chaining those teams on the authorization hot path. Amazon EventBridge is the richer event bus when you need archive/replay and schema discovery; SNS remains the simple pub/sub primitive the exam still uses heavily.

AWS Step Functions orchestrates retries, backoff, catchers, and compensating transactions . A ledger write that fails after SQS delivery becomes a state-machine retry with a human approval state if the compensating path needs it. Step Functions is how you encode the runbook that used to live in a wiki.

New solutions should not introduce a synchronous chain of three Regional APIs. Existing solutions can insert a queue in front of the most failure-prone hop without rewriting the ledger. That is Task 3.4: improve reliability of what already runs.

Auto Scaling, SQS visibility timeouts, and Step Functions retries are self-healing . They still fail when:

The payload is wrong (retrying a double-charge).

Both Regions apply the same bad schema (replication of corruption).

Quotas block scale-out.

Split-brain writers appear after a dual-Region failover without fencing.

Those cases need runbooks : who calls switchover-global-cluster versus unplanned failover, who updates ARC routing controls, who halts writers, who restores from a vault-locked backup instead of promoting a replica that contains ransomware.

DR testing and failure-scenario exercises

Untested RTO is fiction. AWS’s DR whitepaper and DRS product pages both assume non-disruptive drills . Organizational Task 1.3 work is a game-day calendar , blast-radius rules, and a rule that production experiments need stop conditions .

Exercise catalog that maps to this chapter:

AZ impairment — shift traffic (zonal shift) or inject faults (FIS). Remaining AZs must already have capacity.

Regional evacuation — routing control plus database promotion. Measure actual RPO (replica lag) and RTO (DNS TTL + boot + queue drain).

Data corruption / ransomware — restore from a point-in-time backup in a locked or air-gapped vault, not from the infected replica.

Quota exhaustion — attempt scale-out in the DR Region on a normal Tuesday.

Control-plane dependence — try to deploy CloudFormation during a simulated console outage; discover you needed pre-provisioned capacity.

AWS Fault Injection Service (current)

AWS FIS (AWS Fault Injection Service) runs real actions on real resources . AWS tells you to plan in pre-production before production. You author an experiment template : actions , targets , and stop conditions (CloudWatch alarms that abort the experiment). Targets can be selected by resource ID, tags, or filters (including AZ). Selection modes include ALL , COUNT(n) , and PERCENT(n) . Multi-account experiments can target resources in other accounts. Logging can go to CloudWatch Logs or S3. You are billed per action-minute and by number of target accounts (see current FIS pricing; do not memorize a dollar figure AWS can change).

The Scenario Library (console) includes AZ Availability: Power Interruption , which AWS documents as interrupting compute, networking, storage, and managed-service behavior in a chosen AZ—EC2/Auto Scaling capacity errors, subnet disruption, EBS, RDS clusters (writer AZ), ElastiCache power interruption, and related actions. Shared parameters such as affectedAz keep the AZ consistent across actions. Use stop conditions so a runaway experiment does not become an unplanned incident.

FIS is appropriate on SAP-C02 when the stem wants evidence that Multi-AZ works. It is the wrong answer when the stem wants point-in-time restore (that is Backup) or block replication (that is DRS).

Self-healing versus runbook

Situation Prefer automation Prefer a human-gated runbook

One AZ impaired, autoshift enabled, practice runs succeeding Zonal autoshift Confirm downstream single-AZ dependencies

  • Instance host failure Simplified or CloudWatch recovery; Auto Scaling replacement —
  • Queue depth rising, workers healthy Scale workers; Step Functions retries Inspect DLQ before redrive

Unplanned Regional outage, Aurora lag unknown Health-based DNS to read-only Decide managed failover versus waiting; fence old writers

Ransomware / bad deploy replicated everywhere — Restore from vault lock / air-gapped copy; disable replication first

  • Quota or capacity denial in DR Region Alarms Pre-approved quota tickets; alternative Region
  • Payments game day

Existing authorization API still calls the ledger with synchronous HTTPS. Multi-AZ RDS is on. Nobody has failed an AZ on purpose. Task 3.4 plan: (1) put authorize-requests on SQS with a DLQ and idempotent consumers; (2) run FIS AZ Availability: Power Interruption in the staging account with stop conditions; (3) enable zonal shift on the ALB after prescaling; (4) table-top the Regional runbook (ARC routing control + Aurora failover) without executing it in production that day; (5) write the runbook step that says “do not promote a replica if integrity alarms fired—restore from the locked vault.” Self-healing covers the AZ. The runbook covers the day replication lies to you.

Loading diagram... Coupled versus queued authorization path during AZ impairment