Study guide
4.3 Dynamic Scalability Through Caching
AWS Certified CloudOps Engineer – Associate SOA-C03
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Monitoring & Observability Infrastructure
1.1 Configuring Workload Observability 1.2 CloudWatch Agent Architecture & Deployment 1.3 Metric Alarms, Anomaly Detection & Composite Alarms
2 Chapter 2: Alarms, Dashboards & Event-Driven Remediation
2.1 CloudWatch Dashboards & Multi-Account Visibility 2.2 Notification Architecture & Alarm Invocation 2.3 Amazon EventBridge Rules, Event Buses & Troubleshooting 2.4 Automated Remediation with Systems Manager & Lambda
3 Chapter 3: Performance Optimization for Compute, Storage & Databases
3.1 EC2 & Compute Performance Optimization 3.2 Amazon EBS Performance Metrics & Volume Tuning 3.3 S3 Performance Strategies & Data Transfer 3.4 Shared Storage (EFS/FSx) & Database Performance
4 Chapter 4: Compute & Database Scalability and Elasticity
4.1 EC2 Auto Scaling Policies & Elasticity 4.2 Auto Scaling Advanced Operations: Warm Pools & Lifecycle Hooks 4.3 Dynamic Scalability Through Caching 4.4 Managed Database Scaling: RDS, Aurora & DynamoDB
5 Chapter 5: High Availability, Fault Tolerance & Load Balancing
5.1 Elastic Load Balancing (ALB/NLB) Configuration & Troubleshooting 5.2 Route 53 Health Checks & DNS Failover 5.3 Multi-AZ Architecture & Fault-Tolerant Deployments
6 Chapter 6: Backup, Data Protection & Disaster Recovery
6.1 Centralized Backup Automation & AWS Backup 6.2 Database Restore Strategies & RTO/RPO Management 6.3 Storage Versioning, Replication & Immutability 6.4 Disaster Recovery Patterns & Multi-Region Resilience
7 Chapter 7: Infrastructure as Code with CloudFormation & AWS CDK
7.1 CloudFormation Foundations & Resource Modeling 7.2 Managing Stacks, Drift Detection & Change Sets 7.3 Infrastructure as Code with AWS CDK 7.4 Identifying & Remediating Deployment Failures
8 Chapter 8: Multi-Account Provisioning, Containers & Third-Party IaC
8.1 Golden Images with EC2 Image Builder & Amazon ECR 8.2 Multi-Account & Multi-Region Resource Sharing 8.3 Deployment Strategies & Container Deployments 8.4 Third-Party Automation with Terraform & Git Workflows
9 Chapter 9: Systems Manager & Event-Driven Operations Automation
9.1 AWS Systems Manager Core Management & Run Command 9.2 Systems Manager Patch Manager & Session Manager 9.3 Event-Driven Operations Automation
10 Chapter 10: Identity, Access Management & Governance
10.1 Advanced IAM Policies, Roles & Policy Conditions 10.2 Auditing Access with CloudTrail Lake & IAM Access Analyzer 10.3 Multi-Account Security: Organizations, SCPs & Identity Center 10.4 Compliance Enforcement with AWS Config & Conformance Packs
11 Chapter 11: Data Security, Encryption & Vulnerability Management
11.1 Data Classification & Amazon S3 Security 11.2 Encryption at Rest with AWS Key Management Service (KMS) 11.3 Encryption in Transit with AWS Certificate Manager (ACM) 11.4 Secrets Management & Centralized Security Findings
12 Chapter 12: VPC Architecture, Endpoints & Network Protection
12.1 VPC Core Architecture, Subnetting & High Availability NAT 12.2 Private Network Connectivity: VPC Endpoints & Peering 12.3 Network Protection & Edge Security Auditing
13 Chapter 13: DNS, Route 53 Routing & Content Delivery
13.1 DNS Architecture & Route 53 Resolver for Hybrid Clouds 13.2 Route 53 Advanced Routing Policies & Query Logging 13.3 Global Content & Service Distribution: CloudFront & Global Accelerator
14 Chapter 14: Network Troubleshooting, Traffic Analysis & Hybrid Connectivity
14.1 VPC Configuration Troubleshooting & Analysis Tools 14.2 Network Traffic Analysis: VPC Flow Logs & Log Analytics 14.3 CloudFront Caching & Hybrid Connectivity Troubleshooting
4.3 Dynamic Scalability Through Caching
Key Takeaways Explain This Topic AI
A multi-tier caching architecture deploys caching at the edge (CloudFront), in application memory, and at the database layer (ElastiCache) to offload traffic and minimize read latency.
ElastiCache for Valkey/Redis supports advanced data structures, persistence, replication, and horizontal write sharding with Cluster Mode Enabled, whereas Memcached provides a simple, multithreaded in-memory key-value store without replication.
Cache-Aside (Lazy Loading) populates cache on demand with TTL expiration, Write-Through updates cache and database synchronously, and Write-Around avoids cache churn by writing directly to the database.
Cache stampedes (thundering herd) occur when expired keys cause concurrent database surges, mitigated by distributed mutex locking (Redis SETNX) and probabilistic early expiration.
EngineCPUUtilization measures the single-threaded Redis execution core and detects engine saturation that host-level CPUUtilization obscures, while SwapUsage must remain near zero to prevent severe latency spikes.
Last updated: September 2026
4.3 Dynamic Scalability Through Caching
Scaling compute nodes and relational databases horizontally or vertically introduces steep cost curves and architectural bottlenecks. Relational databases face finite connection pools and disk I/O ceilings when hit with intense read spikes. Distributed caching absorbs high-frequency queries, flattens database loads, and delivers sub-millisecond response times. For the CloudOps exam, engineers must master multi-tier caching architectures, Amazon ElastiCache engine internals (Valkey/Redis vs. Memcached), caching patterns, eviction strategies, and CloudWatch performance metrics.
Multi-Tier Caching Architecture
Enterprise cloud architectures deploy caching across three distinct infrastructure tiers:
Edge Tier (Amazon CloudFront) : Content Delivery Network (CDN) with hundreds of Points of Presence (PoPs) globally. CloudFront caches static assets (images, stylesheets, media) and cacheable REST/GraphQL responses using HTTP Cache-Control headers, terminating requests closest to users.
Application Tier (Local In-Memory Cache) : Ephemeral process memory (e.g., local heap maps). Delivers nanosecond lookups but remains isolated per instance or container, creating cache synchronization challenges across auto-scaled fleets.
Database Tier (Amazon ElastiCache) : Centralized, distributed in-memory data store shared across all application instances. Delivers sub-millisecond read latency, shields primary relational databases from query spikes, and provides centralized session management.
Amazon ElastiCache Engine Comparison: Valkey / Redis OSS vs. Memcached
Amazon ElastiCache offers managed engines based on Valkey / Redis OSS and Memcached:
Architectural Feature ElastiCache for Valkey / Redis OSS ElastiCache for Memcached
Data Structures Complex: Strings, Hashes, Lists, Sets, Sorted Sets ( ZSET ), Streams Simple key-value store (strings and serialized objects)
Threading Model Single-threaded core execution loop (I/O offloaded to helper threads) Fully multithreaded (scales vertically on multi-core CPUs)
High Availability Primary-replica replication with Multi-AZ automated failover No native replication; independent standalone nodes
Persistence & Backup Disk persistence via RDB point-in-time snapshots and AOF logs Pure in-memory; no persistence, backup, or restore
Horizontal Scaling Partitioned sharding via Cluster Mode Enabled (up to 500 shards) Client-side consistent hashing across nodes
- Advanced Features Pub/Sub messaging, Geospatial indexes, and Lua scripting Simple key-value caching only
- Redis Cluster Modes
Cluster Mode Disabled : Consists of 1 primary node and up to 5 read replicas in a single shard. All writes route strictly to the primary node. While read capacity scales horizontally across replicas, write throughput is strictly bounded by the single primary , and dataset size cannot exceed a single node's memory.
Cluster Mode Enabled : Partitions data across 1 to 500 shards using 16,384 internal hash slots. Each shard contains 1 primary and up to 5 read replicas, enabling horizontal scaling of both read and write capacity across multi-terabyte datasets.
Caching Patterns & Invalidation Strategies
Applications interact with distributed caches using three primary patterns:
1. Cache-Aside (Lazy Loading)
The application directly orchestrates cache lookups:
Application queries the cache. On a cache hit , it returns data immediately.
On a cache miss , the application queries the database, writes the result to the cache, and returns it to the client.
Trade-offs : Resilient against cache outages (falls back to database); caches only requested keys. However, misses incur three network round-trips, and data can become stale if updated directly in the database.
Mitigation : Configure an explicit Time to Live (TTL) on all keys to force periodic re-fetching.
2. Write-Through
The application writes data to the cache and the persistent database simultaneously:
Trade-offs : Cache data is always fresh, eliminating read-time database misses. However, write latency increases because writes must succeed across two systems, and memory can be flooded with data that is rarely read. Usually combined with eviction policies or TTLs.
3. Write-Around
Data is written directly to the database without populating the cache. Data enters the cache only upon a subsequent cache miss via Cache-Aside. This prevents cache pollution from high-volume write operations that are seldom read (e.g., historical audit logs).
Eviction Policies & Thundering Herd Mitigation
When memory reaches maxmemory , ElastiCache invokes an eviction policy to remove existing keys:
volatile-lru / allkeys-lru : Evicts Least Recently Used keys (either with explicit TTL or across all keys).
volatile-lfu / allkeys-lfu : Evicts Least Frequently Used keys.
noeviction : Returns an out-of-memory error on write operations when memory is full.
Mitigating Cache Stampede (Thundering Herd)
A cache stampede occurs when a heavily queried key expires, causing thousands of concurrent threads to hit the backend database simultaneously on cache miss. CloudOps engineers mitigate stampedes using:
Distributed Mutex Locking : Threads attempt to acquire a distributed lock in Redis (using SET key value NX PX 10000 ). Only the single thread obtaining the lock queries the database and refreshes the cache; all other threads wait or consume stale cached values.
Probabilistic Early Expiration (XFetch) : The application calculates randomized early expiration windows, refreshing keys in the background before the hard TTL expires.
Monitoring ElastiCache with CloudWatch
CloudOps engineers must track critical ElastiCache CloudWatch metrics:
EngineCPUUtilization : Critical for Redis / Valkey . Because Redis executes commands in a single-threaded loop, a 4-vCPU instance shows 25% host CPUUtilization when its Redis engine core is 100% saturated. Always monitor EngineCPUUtilization to detect Redis bottlenecks. For Memcached, host CPUUtilization accurately reflects multi-threaded CPU usage.
SwapUsage : Indicates OS memory paging. In an in-memory database, swapping causes severe latency spikes (from microseconds to milliseconds). SwapUsage must not exceed 50 MB .
Evictions : Spikes indicate memory exhaustion where valid keys are being dropped, requiring cluster scale-up or shard addition.
CurrConnections : Spikes indicate connection pooling failures or application connection leaks.
ElastiCache supports Application Auto Scaling to automatically add or remove read replicas or scale shards based on EngineCPUUtilization and memory metrics.