ExamFinch.com My Study

Study guide

1.2 CloudWatch Agent Architecture & Deployment

Practice this exam

AWS Certified CloudOps Engineer – Associate SOA-C03

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Monitoring & Observability Infrastructure

1.1 Configuring Workload Observability 1.2 CloudWatch Agent Architecture & Deployment 1.3 Metric Alarms, Anomaly Detection & Composite Alarms

2 Chapter 2: Alarms, Dashboards & Event-Driven Remediation

2.1 CloudWatch Dashboards & Multi-Account Visibility 2.2 Notification Architecture & Alarm Invocation 2.3 Amazon EventBridge Rules, Event Buses & Troubleshooting 2.4 Automated Remediation with Systems Manager & Lambda

3 Chapter 3: Performance Optimization for Compute, Storage & Databases

3.1 EC2 & Compute Performance Optimization 3.2 Amazon EBS Performance Metrics & Volume Tuning 3.3 S3 Performance Strategies & Data Transfer 3.4 Shared Storage (EFS/FSx) & Database Performance

4 Chapter 4: Compute & Database Scalability and Elasticity

4.1 EC2 Auto Scaling Policies & Elasticity 4.2 Auto Scaling Advanced Operations: Warm Pools & Lifecycle Hooks 4.3 Dynamic Scalability Through Caching 4.4 Managed Database Scaling: RDS, Aurora & DynamoDB

5 Chapter 5: High Availability, Fault Tolerance & Load Balancing

5.1 Elastic Load Balancing (ALB/NLB) Configuration & Troubleshooting 5.2 Route 53 Health Checks & DNS Failover 5.3 Multi-AZ Architecture & Fault-Tolerant Deployments

6 Chapter 6: Backup, Data Protection & Disaster Recovery

6.1 Centralized Backup Automation & AWS Backup 6.2 Database Restore Strategies & RTO/RPO Management 6.3 Storage Versioning, Replication & Immutability 6.4 Disaster Recovery Patterns & Multi-Region Resilience

7 Chapter 7: Infrastructure as Code with CloudFormation & AWS CDK

7.1 CloudFormation Foundations & Resource Modeling 7.2 Managing Stacks, Drift Detection & Change Sets 7.3 Infrastructure as Code with AWS CDK 7.4 Identifying & Remediating Deployment Failures

8 Chapter 8: Multi-Account Provisioning, Containers & Third-Party IaC

8.1 Golden Images with EC2 Image Builder & Amazon ECR 8.2 Multi-Account & Multi-Region Resource Sharing 8.3 Deployment Strategies & Container Deployments 8.4 Third-Party Automation with Terraform & Git Workflows

9 Chapter 9: Systems Manager & Event-Driven Operations Automation

9.1 AWS Systems Manager Core Management & Run Command 9.2 Systems Manager Patch Manager & Session Manager 9.3 Event-Driven Operations Automation

10 Chapter 10: Identity, Access Management & Governance

10.1 Advanced IAM Policies, Roles & Policy Conditions 10.2 Auditing Access with CloudTrail Lake & IAM Access Analyzer 10.3 Multi-Account Security: Organizations, SCPs & Identity Center 10.4 Compliance Enforcement with AWS Config & Conformance Packs

11 Chapter 11: Data Security, Encryption & Vulnerability Management

11.1 Data Classification & Amazon S3 Security 11.2 Encryption at Rest with AWS Key Management Service (KMS) 11.3 Encryption in Transit with AWS Certificate Manager (ACM) 11.4 Secrets Management & Centralized Security Findings

12 Chapter 12: VPC Architecture, Endpoints & Network Protection

12.1 VPC Core Architecture, Subnetting & High Availability NAT 12.2 Private Network Connectivity: VPC Endpoints & Peering 12.3 Network Protection & Edge Security Auditing

13 Chapter 13: DNS, Route 53 Routing & Content Delivery

13.1 DNS Architecture & Route 53 Resolver for Hybrid Clouds 13.2 Route 53 Advanced Routing Policies & Query Logging 13.3 Global Content & Service Distribution: CloudFront & Global Accelerator

14 Chapter 14: Network Troubleshooting, Traffic Analysis & Hybrid Connectivity

14.1 VPC Configuration Troubleshooting & Analysis Tools 14.2 Network Traffic Analysis: VPC Flow Logs & Log Analytics 14.3 CloudFront Caching & Hybrid Connectivity Troubleshooting

1.2 CloudWatch Agent Architecture & Deployment

Key Takeaways Explain This Topic AI

The unified CloudWatch Agent captures operating system metrics invisible to the hypervisor, such as memory utilization, disk space, swap usage, and TCP connections

Agent configuration is defined via amazon-cloudwatch-agent.json, generated via wizard, and centrally stored in AWS Systems Manager Parameter Store

SSM Run Command executes the AmazonCloudWatch-ManageAgent document to deploy and refresh agent configurations across EC2 fleets without SSH access

Amazon ECS Container Insights captures task and cluster resource utilization via Embedded Metric Format (EMF) logs in CloudWatch Logs

Amazon EKS Container Insights utilizes CloudWatch Agent and ADOT DaemonSets for pod and persistent volume metrics, paired with Fluent Bit for container log routing

Last updated: September 2026

CloudWatch Agent Architecture & Hypervisor Visibility Boundary

Operating infrastructure across Amazon EC2, Amazon ECS, and Amazon EKS requires telemetry at both hypervisor and guest operating system layers. Early AWS setups relied on separate tools: the legacy CloudWatch Logs agent ( awslogs ) and python/perl scripts ( mon-scripts ). AWS unified these into the modern Amazon CloudWatch Agent , which runs on Linux and Windows to collect system metrics and log streams through a single daemon.

Understanding the hypervisor boundary is essential for CloudOps: +-------------------------------------------------------------+

  • | GUEST OPERATING SYSTEM (Requires CloudWatch Agent) |
  • | - Available & Used Memory (mem_used_percent) |
  • | - Filesystem & Inode Utilization (disk_used_percent) |
  • | - Swap Usage (swap_used_percent) |
  • | - Active TCP Sockets & State Counters (netstat_*) |
  • | - Process-Level CPU & RSS Memory (procstat plugin) |
  • +-------------------------------------------------------------+
  • ==================== HYPERVISOR BOUNDARY ======================
  • +-------------------------------------------------------------+
  • | AWS HYPERVISOR (Nitro / Xen - Default EC2 Metrics) |
  • | - CPU Utilization (CPUUtilization) |
  • | - Hypervisor Network Traffic (NetworkIn, NetworkOut) |
  • | - Instance Store Disk I/O (DiskReadBytes, DiskWriteBytes) |
  • | - Physical Hardware Status (StatusCheckFailed_System) |
  • +-------------------------------------------------------------+

The hypervisor manages physical CPUs, virtualized NICs, and storage attachments, providing metrics like CPUUtilization with zero in-guest software. However, the hypervisor cannot inspect guest kernel memory pages, virtual memory allocations, or filesystem directory structures. Memory and disk metrics remain invisible without the CloudWatch Agent.

Guest Operating System Metrics Collection on EC2

The CloudWatch Agent collects five core categories of guest metrics:

Memory Metrics : The hypervisor cannot detect memory exhaustion. The agent captures mem_used_percent , mem_available , mem_used , and mem_total . Auto-scaling and paging alarms typically target mem_used_percent .

Disk and Filesystem Metrics : Gathers storage utilization per mount point and filesystem type, including disk_used_percent , disk_used , disk_free , and disk_inodes_used . Inode monitoring prevents file-creation failures when storage blocks remain available.

Swap Metrics : Virtual memory paging counters include swap_used_percent and swap_used . High swap usage coupled with elevated disk I/O reveals severe memory exhaustion and thrashing.

Network Socket Metrics ( netstat ) : Gathers TCP connection states, such as netstat_tcp_established , netstat_tcp_time_wait , and netstat_tcp_close_wait , highlighting socket exhaustion or unclosed database connections.

Process Metrics ( procstat ) : The procstat plugin monitors individual system processes using pid_file , exe name, or regex pattern . It publishes metrics like procstat_cpu_usage and procstat_memory_rss for daemons such as NGINX, Apache, or MySQL.

Centralized Configuration Management with Systems Manager

The CloudWatch Agent relies on a single JSON configuration: amazon-cloudwatch-agent.json . The configuration schema divides into three primary stanzas:

"agent" : Controls operational parameters, collection intervals, and runtime OS user ( run_as_user ).

"metrics" : Configures metric plugins under "metrics_collected" , defining collection intervals, units, and aggregation dimensions (such as InstanceId and AutoScalingGroupName ).

"logs" : Specifies log files under "logs_collected" , mapping source file paths to CloudWatch log groups and streams.

Configuration Creation & Parameter Store Integration

Administrators generate baseline configurations using the interactive wizard: sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

In enterprise environments, configurations are stored in AWS Systems Manager (SSM) Parameter Store as standard String parameters (e.g., AmazonCloudWatch-linux or AmazonCloudWatch-windows ).

Fleet-Wide Deployment via SSM Run Command

Deploying or updating the agent across hundreds of EC2 instances executes without SSH access using SSM Run Command with the document AmazonCloudWatch-ManageAgent :

  • action : configure
  • mode : ec2
  • optionalConfigurationSource : ssm
  • optionalConfigurationLocation : AmazonCloudWatch-linux
  • optionalRestart : yes

IAM Role Prerequisite : Every target EC2 instance requires an IAM instance profile with the managed policy CloudWatchAgentServerPolicy (granting permissions for cloudwatch:PutMetricData , logs:PutLogEvents , logs:CreateLogStream , and ssm:GetParameter ).

Container Observability: Amazon ECS and Amazon EKS

Containerized architectures require specialized observability patterns:

Amazon ECS Container Insights

Amazon ECS Container Insights delivers monitoring for ECS clusters, services, and tasks. It is activated at the cluster level via CLI or console: aws ecs update-cluster-settings --cluster ProductionCluster --settings name=containerInsights,value=enabled

ECS automatically collects task-level CPU and memory utilization, network I/O, and storage metrics without manual agent configuration. Metrics are emitted as Embedded Metric Format (EMF) logs into the log group /aws/ecs/containerinsights/<cluster-name>/performance . For application logs, task definitions configure the awslogs driver, streaming stdout/stderr to CloudWatch Logs.

Amazon EKS Container Insights with Enhanced Observability

In Kubernetes environments on Amazon EKS:

CloudWatch Agent DaemonSet : Deployed across worker nodes alongside the AWS Distro for OpenTelemetry (ADOT) collector within the amazon-cloudwatch namespace. Enhanced observability captures pod-level CPU/memory utilization, control plane performance (API server latency, etcd metrics), and persistent volume storage metrics ( pod_volume_fs_used_percent ) via the EBS CSI driver.

AWS for Fluent Bit DaemonSet : High-performance log processor deployed as a Kubernetes DaemonSet. Fluent Bit parses container logs, enriches them with Kubernetes metadata (namespace, pod name), and routes them to dedicated CloudWatch log groups:

  • /aws/eks/<cluster-name>/application : Application container logs
  • /aws/eks/<cluster-name>/dataplane : Kubelet, containerd, and node system logs
  • /aws/eks/<cluster-name>/host : Operating system audit and auth logs
  • Agent Management & Common Troubleshooting Traps

Administrators manage the agent locally with amazon-cloudwatch-agent-ctl :

Start agent with SSM config: sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:AmazonCloudWatch-linux

  • Check runtime status: sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
  • Diagnostic log path: /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log
  • Common Configuration Traps

Private Subnet Connectivity : Instances in private subnets without NAT require Interface VPC Endpoints (AWS PrivateLink) for CloudWatch ( monitoring ), CloudWatch Logs ( logs ), and Systems Manager ( ssm , ec2messages , ssmmessages ).

Missing IAM Permissions : Omitting CloudWatchAgentServerPolicy produces AccessDenied errors when publishing metrics or fetching SSM parameters.

Invalid Filesystem Paths : Specifying non-existent mount points in the disk plugin aborts startup or prevents disk metric emission.