Study guide

2.4 Automated Remediation with Systems Manager & Lambda

AWS Certified CloudOps Engineer – Associate SOA-C03

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Monitoring & Observability Infrastructure

1.1 Configuring Workload Observability 1.2 CloudWatch Agent Architecture & Deployment 1.3 Metric Alarms, Anomaly Detection & Composite Alarms

2 Chapter 2: Alarms, Dashboards & Event-Driven Remediation

2.1 CloudWatch Dashboards & Multi-Account Visibility 2.2 Notification Architecture & Alarm Invocation 2.3 Amazon EventBridge Rules, Event Buses & Troubleshooting 2.4 Automated Remediation with Systems Manager & Lambda

3 Chapter 3: Performance Optimization for Compute, Storage & Databases

3.1 EC2 & Compute Performance Optimization 3.2 Amazon EBS Performance Metrics & Volume Tuning 3.3 S3 Performance Strategies & Data Transfer 3.4 Shared Storage (EFS/FSx) & Database Performance

4 Chapter 4: Compute & Database Scalability and Elasticity

4.1 EC2 Auto Scaling Policies & Elasticity 4.2 Auto Scaling Advanced Operations: Warm Pools & Lifecycle Hooks 4.3 Dynamic Scalability Through Caching 4.4 Managed Database Scaling: RDS, Aurora & DynamoDB

5 Chapter 5: High Availability, Fault Tolerance & Load Balancing

5.1 Elastic Load Balancing (ALB/NLB) Configuration & Troubleshooting 5.2 Route 53 Health Checks & DNS Failover 5.3 Multi-AZ Architecture & Fault-Tolerant Deployments

6 Chapter 6: Backup, Data Protection & Disaster Recovery

6.1 Centralized Backup Automation & AWS Backup 6.2 Database Restore Strategies & RTO/RPO Management 6.3 Storage Versioning, Replication & Immutability 6.4 Disaster Recovery Patterns & Multi-Region Resilience

7 Chapter 7: Infrastructure as Code with CloudFormation & AWS CDK

7.1 CloudFormation Foundations & Resource Modeling 7.2 Managing Stacks, Drift Detection & Change Sets 7.3 Infrastructure as Code with AWS CDK 7.4 Identifying & Remediating Deployment Failures

8 Chapter 8: Multi-Account Provisioning, Containers & Third-Party IaC

8.1 Golden Images with EC2 Image Builder & Amazon ECR 8.2 Multi-Account & Multi-Region Resource Sharing 8.3 Deployment Strategies & Container Deployments 8.4 Third-Party Automation with Terraform & Git Workflows

9 Chapter 9: Systems Manager & Event-Driven Operations Automation

9.1 AWS Systems Manager Core Management & Run Command 9.2 Systems Manager Patch Manager & Session Manager 9.3 Event-Driven Operations Automation

10 Chapter 10: Identity, Access Management & Governance

10.1 Advanced IAM Policies, Roles & Policy Conditions 10.2 Auditing Access with CloudTrail Lake & IAM Access Analyzer 10.3 Multi-Account Security: Organizations, SCPs & Identity Center 10.4 Compliance Enforcement with AWS Config & Conformance Packs

11 Chapter 11: Data Security, Encryption & Vulnerability Management

11.1 Data Classification & Amazon S3 Security 11.2 Encryption at Rest with AWS Key Management Service (KMS) 11.3 Encryption in Transit with AWS Certificate Manager (ACM) 11.4 Secrets Management & Centralized Security Findings

12 Chapter 12: VPC Architecture, Endpoints & Network Protection

12.1 VPC Core Architecture, Subnetting & High Availability NAT 12.2 Private Network Connectivity: VPC Endpoints & Peering 12.3 Network Protection & Edge Security Auditing

13 Chapter 13: DNS, Route 53 Routing & Content Delivery

13.1 DNS Architecture & Route 53 Resolver for Hybrid Clouds 13.2 Route 53 Advanced Routing Policies & Query Logging 13.3 Global Content & Service Distribution: CloudFront & Global Accelerator

14 Chapter 14: Network Troubleshooting, Traffic Analysis & Hybrid Connectivity

14.1 VPC Configuration Troubleshooting & Analysis Tools 14.2 Network Traffic Analysis: VPC Flow Logs & Log Analytics 14.3 CloudFront Caching & Hybrid Connectivity Troubleshooting

2.4 Automated Remediation with Systems Manager & Lambda

Key Takeaways Explain This Topic AI

Automated remediation loops integrate detection (CloudWatch Alarms, EventBridge Rules, AWS Config Rules) with execution (SSM Automation runbooks or AWS Lambda) and audit trails (AWS CloudTrail).

SSM Automation runbooks provide pre-built, long-running, and OS-aware administrative workflows (e.g., AWS-RestartEC2Instance, AWS-EnableS3BucketEncryption) with native approval gates (aws:approve).

SSM Automation documents require schemaVersion 0.3, an AutomationAssumeRole with appropriate IAM permissions, and step action plugins like aws:executeAwsApi and aws:changeInstanceState.

AWS Config rules provide continuous compliance monitoring that triggers SSM Automation remediation automatically, mapping the non-compliant resource ID to the runbook parameter.

Production incident response patterns utilize automated runbooks to quarantine compromised EC2 instances into isolation security groups, capture volatile memory and EBS snapshots, and detach instances from Auto Scaling groups to prevent premature termination.

Last updated: September 2026

2.4 Automated Remediation with Systems Manager & Lambda

Automated remediation transforms static monitoring into self-healing infrastructure. When operational incidents, resource failures, or compliance violations occur, relying on human engineers to diagnose and execute corrective actions introduces latency, operational overhead, and human error. AWS provides two primary compute engines for automated remediation: AWS Systems Manager (SSM) Automation and AWS Lambda . For the CloudOps Associate exam, engineers must know how to choose between these engines, design multi-step runbooks, implement approval gates, integrate with AWS Config rules, and execute production incident response workflows.

Remediation Workflow Architecture: Choosing SSM Automation vs. Lambda

An automated remediation loop follows three distinct operational phases:

Detection : A CloudWatch Alarm trips into the ALARM state, an EventBridge rule detects a state-change or API call, or an AWS Config rule flags a resource as non-compliant.

Orchestration & Execution : The detection mechanism triggers either an SSM Automation runbook or an AWS Lambda function.

Audit & Verification : Every remediation action must be logged in AWS CloudTrail, Systems Manager execution history, and CloudWatch Logs to ensure traceability and verify successful resolution.

Choosing the appropriate execution engine is a common exam theme:

Architectural Criterion AWS Systems Manager Automation AWS Lambda

Primary Use Case Multi-service AWS API orchestration, fleet operations, OS-level administration Custom algorithmic business logic, complex data transformations, rapid micro-actions

Code Maintenance Low (declarative YAML/JSON documents; hundreds of pre-built AWS runbooks) Higher (requires authoring, testing, packaging, and maintaining runtime dependencies)

Execution Duration Long-running workflows (days or weeks; supports pauses and approval gates) Hard limit of 15 minutes (900 seconds) per invocation

OS-Level Execution Native integration with SSM Agent via Run Command plugins Cannot execute commands directly inside private OS instances without bastion/network proxies

Human Approval Gates Native support via the aws:approve action plugin Requires custom Step Functions state machine or external webhooks

As a general CloudOps rule of thumb: prefer AWS-managed SSM Automation runbooks first . Only introduce AWS Lambda when custom parsing, external third-party API integration, or sub-second latency is strictly necessary.

SSM Automation Runbooks: Structure & Execution

An SSM Automation runbook is a declarative document (defined in YAML or JSON) that specifies the tasks executed by Systems Manager. AWS provides dozens of predefined, battle-tested runbooks, including:

AWS-RestartEC2Instance : Safely reboots an EC2 instance.

AWS-StopEC2Instance : Stops an instance to halt runaway costs or isolate workloads.

AWS-CreateSnapshot : Generates point-in-time EBS volume snapshots prior to maintenance.

AWS-PublishSNSNotification : Broadcasts incident updates to operational channels.

AWS-AttachEBSVolume : Automatically attaches replacement storage volumes.

Document Structure & The Automation Assume Role

An Automation document consists of several core sections:

schemaVersion : Must be specified as 0.3 to support modern branching, looping, and multi-step workflows.

assumeRole : The ARN of the IAM role that Systems Manager assumes to perform actions ( AutomationAssumeRole ). This role must have a trust policy allowing ssm.amazonaws.com and identity policies granting the specific AWS API actions executed in the runbook.

parameters : Defines inputs passed at runtime (e.g., InstanceId , SnapshotId ), along with types, descriptions, and defaults.

mainSteps : An ordered sequence of steps using SSM action plugins.

outputs : Values exported from steps for subsequent runbooks or parent workflows.

A production-grade custom runbook leverages action plugins such as aws:executeAwsApi (making direct AWS API calls), aws:waitForAwsResourceProperty (polling until resources enter a desired state), and conditional branching ( onFailure , nextStep ): description: "Diagnose and reboot unresponsive EC2 instance"

  • schemaVersion: "0.3"
  • assumeRole: "arn:aws:iam::123456789012:role/SSMAutomationRole"

parameters:

InstanceId:

  • type: "String"
  • description: "Target EC2 Instance to remediate"

mainSteps:

  • name: "CreateSafetySnapshot"
  • action: "aws:executeAwsApi"

inputs:

  • Service: "ec2"
  • Api: "CreateSnapshot"
  • VolumeId: "{{GetVolume.VolumeId}}"
  • Description: "Pre-remediation snapshot"
  • nextStep: "RestartInstance"
  • onFailure: "Abort"
  • name: "RestartInstance"
  • action: "aws:changeInstanceState"

inputs:

InstanceIds:

  • "{{InstanceId}}"
  • CheckStateOnly: false
  • DesiredState: "reboot"
  • nextStep: "WaitForHealthy"
  • name: "WaitForHealthy"
  • action: "aws:waitForAwsResourceProperty"
  • timeoutSeconds: 300

inputs:

  • Service: "ec2"
  • Api: "DescribeInstanceStatus"

InstanceIds:

  • "{{InstanceId}}"
  • PropertySelector: "$.InstanceStatuses[0].InstanceStatus.Status"

DesiredValues:

  • "ok"
  • isEnd: true
  • Manual Approval Steps ( aws:approve )

In mission-critical or high-risk environments, full automation may be restricted by compliance policies. SSM Automation supports hybrid workflows using the aws:approve action plugin. When reached, execution pauses, dispatches an approval notification via Amazon SNS to designated Approvers (IAM users or roles), and waits for a human operator to issue an Approve or Reject API call before proceeding to potentially destructive steps (such as instance termination or database failover).

Integrating Remediation with AWS Config Rules

AWS Config continuously evaluates AWS resource configurations against desired baselines. When a managed or custom Config rule detects non-compliance, it can initiate automatic remediation via SSM Automation.

To configure automatic remediation:

Select the AWS Config rule (e.g., s3-bucket-server-side-encryption-enabled or restricted-ssh ).

Attach an SSM Automation remediation action (e.g., AWS-EnableS3BucketEncryption ).

Configure the Resource ID parameter to map the non-compliant resource ID emitted by Config directly to the runbook's target parameter (e.g., BucketName ).

Specify the AutomationAssumeRole ARN with permissions to modify the target resource.

Set the execution mode to Automatic and define retry attempts (e.g., retry 5 times every 60 seconds).

As soon as a developer creates an unencrypted S3 bucket, AWS Config detects non-compliance within seconds and triggers the runbook, applying default SSE-S3 or KMS encryption without human intervention.

  • Operational Scenarios: Self-Healing & Incident Response
  • 1. Application Daemon Recovery via SSM Run Command

When a CloudWatch metric alarm detects that a critical daemon (e.g., httpd or nginx ) has stopped, an EventBridge rule invokes an SSM Automation runbook. The runbook uses the aws:runCommand plugin to execute systemctl restart nginx via the SSM Agent on the instance, avoiding SSH key management and private network access restrictions.

2. Compromised Instance Quarantine & Digital Forensics

When AWS GuardDuty detects that an EC2 instance is communicating with a known command-and-control server:

EventBridge intercepts the GuardDuty finding and invokes an incident response SSM Automation runbook.

The runbook immediately detaches the instance from its Auto Scaling group (or puts it in standby) to prevent the ASG from terminating it.

The runbook replaces the instance's security groups with an Isolation Security Group that denies all inbound and outbound traffic, severing network access.

The runbook takes point-in-time EBS volume snapshots of all attached disks for forensic analysis.

It triggers volatile memory capture via the SSM Agent before stopping the instance.

All actions are logged immutably in CloudTrail for legal and compliance auditability.