ExamFinch.com My Study

Study guide

13.2 Event-Driven Infrastructure Self-Healing with Lambda & Step Functions

Practice this exam

AWS Certified DevOps Engineer – Professional Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: CI/CD Pipeline Architecture & Source Integration

1.1 Multi-Stage & Multi-Account CodePipeline Design 1.2 CodeBuild Projects, Build Environments & Pipeline Secrets 1.3 Source Control Workflows, Webhooks & Branching Models

2 Chapter 2: Automated Testing, Test Gates & Quality Verification

2.1 Automated Unit, Integration & Acceptance Testing in Pipelines 2.2 Load Testing, Performance Benchmarking & Synthetic Canaries 2.3 Security Scanning, SAST/DAST & Approval Gates

3 Chapter 3: Artifact Management & Multi-Platform Deployment Strategies

3.1 Artifact Lifecycle Management with CodeArtifact, ECR & S3 3.2 Automated Image Baking with EC2 Image Builder 3.3 In-Place, Rolling & Immutable Deployments on EC2 3.4 Blue/Green & Canary Deployments for ECS, EKS & Lambda

4 Chapter 4: CloudFormation, CDK & Reusable IaC Components

4.1 Advanced CloudFormation: Nested Stacks, Change Sets & Custom Resources 4.2 Cross-Account & Multi-Region StackSets & Drift Detection 4.3 Modern IaC: AWS CDK, SAM & Service Catalog Governance

5 Chapter 5: Multi-Account Architecture, Organizations & Control Tower

5.1 AWS Organizations Hierarchy, OUs & Account Provisioning 5.2 Service Control Policies (SCPs) & Multi-Account Guardrails 5.3 AWS Control Tower, Account Factory & Landing Zone Automation

6 Chapter 6: Systems Manager & Large-Scale Fleet Configuration

6.1 SSM Agent, Hybrid Activation & Session Manager 6.2 Systems Manager Run Command, State Manager & Associations 6.3 Patch Manager, Parameter Store vs Secrets Manager & AppConfig

7 Chapter 7: High Availability & Multi-Region Resiliency

7.1 Designing Resilient Multi-AZ & Multi-Region Architectures 7.2 Route 53 Traffic Flow, Health Checks & DNS Failover 7.3 Data Replication: Aurora Global DB, DynamoDB Global Tables & S3 CRR

8 Chapter 8: Scalability, Auto Scaling & Disaster Recovery Automation

8.1 EC2 Auto Scaling Policies, Lifecycle Hooks & Warm Pools 8.2 Container & Serverless Scalability: ECS Capacity Providers, EKS & API Gateway 8.3 Disaster Recovery Strategies: RTO, RPO & Architecture Tiers 8.4 Automated Backup & Recovery with AWS Backup & Elastic Disaster Recovery

9 Chapter 9: CloudWatch Metrics, Agent Collection & Log Aggregation

9.1 CloudWatch Metrics, Dimensions, Metric Math & High Resolution 9.2 Unified CloudWatch Agent & Custom Metric Collection 9.3 Centralized Log Aggregation, Retention & Cross-Account Subscriptions

10 Chapter 10: Log Analysis, Dashboards & Distributed Tracing with X-Ray

10.1 CloudWatch Logs Insights, Filter Syntax & Query Optimization 10.2 CloudWatch Dashboards, Composite Alarms & Anomaly Detection 10.3 Distributed Tracing with AWS X-Ray & Service Maps

11 Chapter 11: Automated Observability & Event-Driven Monitoring

11.1 Application & Infrastructure Health Checks with Route 53 & ALB 11.2 Centralized Observability with Amazon OpenSearch & Athena 11.3 Synthetic Monitoring with CloudWatch Synthetics Canaries

12 Chapter 12: Event-Driven Incident Detection & Notification Pipelines

12.1 AWS Health, CloudTrail Events & Operational Notification 12.2 EventBridge Event Routing, Patterns & Multi-Account Buses 12.3 Asynchronous Event Processing: SNS, SQS Queues & Dead-Letter Handling

13 Chapter 13: Automated Remediation & Deployment Troubleshooting

13.1 AWS Config Managed Rules & Automated Remediation with SSM 13.2 Event-Driven Infrastructure Self-Healing with Lambda & Step Functions 13.3 Troubleshooting CI/CD Pipeline & Deployment Failures 13.4 Root Cause Analysis for Auto Scaling & Container Failures

14 Chapter 14: Identity, Access Management & Secrets at Scale

14.1 IAM Policy Evaluation Logic, NotAction & Explicit Denies 14.2 IAM Permissions Boundaries & Delegated Administration 14.3 IAM Identity Center, Cross-Account Roles & STS Session Policies 14.4 Automated Secrets Management & Secret Rotation with Secrets Manager

15 Chapter 15: Security Automation, Data Protection & Compliance Auditing

15.1 Network Security Automation: Security Groups, NACLs, Network Firewall & WAF 15.2 Cryptographic Controls & Key Management with AWS KMS & ACM 15.3 Automated Threat Detection: GuardDuty, Inspector & Security Hub 15.4 Continuous Compliance Auditing & Log Integrity with CloudTrail & Config

13.2 Event-Driven Infrastructure Self-Healing with Lambda & Step Functions

Key Takeaways Explain This Topic AI

Event-driven self-healing couples real-time telemetry (CloudTrail API mutations, GuardDuty findings, AWS Health events, CloudWatch alarms) with automated compute handlers via Amazon EventBridge.

AWS Lambda is suited to short-running stateless fixes, while AWS Step Functions coordinates stateful, multi-step workflows with branching, retries, parallel actions, and durable waits.

Step Functions state machine resilience is defined through declarative Retry blocks (configuring ErrorEquals, IntervalSeconds, MaxAttempts, BackoffRate) and Catch blocks for graceful failure routing.

Human-in-the-loop workflows utilize task tokens (.waitForTaskToken), pausing execution until an operator or approval system invokes SendTaskSuccess or SendTaskFailure.

Existing AWS Systems Manager Incident Manager customers can use Response Plans, Escalation Plans, contacts, and SSM runbooks, but the service has been closed to new customers since November 7, 2025 and receives no new features.

Last updated: September 2026

Event-Driven Infrastructure Self-Healing Architecture

Traditional IT operations rely on human operators receiving alerts, reading documentation, and manually executing runbooks. In modern cloud-native organizations, self-healing architectures eliminate manual intervention by continuously monitoring infrastructure telemetry, detecting drift or security compromises, and executing automated remediation pipelines within seconds.

Ingestion, Decision, and Remediation Topology

Self-healing systems follow a standardized three-tier topology:

Event Ingestion Tier : Real-time event sources capture control-plane and data-plane mutations:

Amazon EventBridge : Captures AWS CloudTrail management API calls (e.g., ec2:AuthorizeSecurityGroupIngress , s3:PutBucketAcl ), AWS Health events, Amazon GuardDuty findings, and AWS Config compliance changes.

Amazon CloudWatch Alarms : Detects metric breaches such as sustained high CPU, elevated HTTP 5xx error rates, or memory starvation.

Decision & Routing Tier : EventBridge rules evaluate event payloads against granular JSON event patterns, filtering out noise and dispatching matching payloads to appropriate targets.

Remediation Execution Tier : Dedicated serverless compute engines execute self-healing actions:

AWS Lambda : Optimized for fast, single-step, stateless remediation.

AWS Step Functions : Optimized for complex, multi-step, stateful orchestration requiring exponential backoff, parallel execution, rollback logic, or human approval gates.

AWS Systems Manager (SSM) Automation : Optimized for OS-level and infrastructure runbook execution.

Lambda vs. Step Functions vs. SSM Automation

  • Capability AWS Lambda AWS Step Functions SSM Automation
  • Execution Paradigm Stateless function Stateful state machine Managed workflow document
  • Execution Limit 15 minutes Up to 1 year (Standard) Up to 30 days
  • Error Handling Imperative code (try/catch) Declarative ( Retry , Catch ) Step-level onFailure actions

Human-in-the-Loop Requires polling or custom DB Native Task Tokens ( .waitForTaskToken ) Native approval steps ( aws:approve )

Parallelism Must manage threading in code Native Parallel and Map states Concurrent execution across targets

Ideal Use Case Instant single-action fixes (revoke SG rule, quarantine API key) Complex, multi-stage remediation (isolate host, snapshot, notify, terminate) OS patching, instance reboots, software maintenance

Loading diagram... Comprehensive Event-Driven Self-Healing & Incident Management Pipeline

Step Functions State Machine Orchestration Mechanics

When remediation involves multiple sequential or concurrent steps—such as containing a compromised EC2 instance without losing forensic evidence— AWS Step Functions provides stateful, resilient coordination.

Declarative Error Handling: Retry and Catch

Distributed remediation tasks often encounter transient errors (e.g., AWS API throttling, temporary network timeouts, or eventual consistency delays). Step Functions eliminates brittle retry code by providing declarative error handling: {

  • "Type": "Task",
  • "Resource": "arn:aws:states:::lambda:invoke",
  • "Parameters": {
  • "FunctionName": "arn:aws:lambda:us-east-1:123456789012:function:SnapshotEBSVolume",
  • "Payload.$": "$"
  • },
  • "Retry": [
  • {
  • "ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException", "EC2.ThrottlingException"],
  • "IntervalSeconds": 2,
  • "MaxAttempts": 4,
  • "BackoffRate": 2.0
  • }
  • ],
  • "Catch": [
  • {
  • "ErrorEquals": ["States.ALL"],
  • "ResultPath": "$.errorDetails",
  • "Next": "NotifySecurityOperationsDeadLetter"
  • }
  • ],
  • "Next": "TagForensicSnapshot"
  • }

ErrorEquals : Defines which error codes trigger the block. Built-in error names include States.ALL (wildcard matching all errors), States.Timeout , States.TaskFailed , and States.Permissions .

IntervalSeconds : The initial delay in seconds before the first retry attempt (e.g., 2 seconds).

MaxAttempts : The maximum number of retry attempts before giving up and evaluating catch blocks (e.g., 4 attempts).

BackoffRate : The exponential multiplier. A value of 2.0 with an initial interval of 2 seconds results in retry waits of 2s, 4s, 8s, and 16s.

Catch : If all retries fail, execution routes to the state named in Next , injecting error data into the state payload via ResultPath .

Parallel Execution for Accelerated Remediation

In security incident response, speed of containment is vital. The Parallel state executes multiple discrete tasks concurrently: {

  • "Type": "Parallel",
  • "Branches": [
  • {
  • "StartAt": "IsolateNetworkTraffic",
  • "States": {
  • "IsolateNetworkTraffic": {
  • "Type": "Task",
  • "Resource": "arn:aws:states:::aws-sdk:ec2:modifyInstanceAttribute",
  • "Parameters": {
  • "InstanceId.$": "$.detail.instanceId",
  • "Groups": ["sg-isolated-forensics-only"]
  • },
  • "End": true
  • }
  • }
  • },
  • {
  • "StartAt": "CreateVolumeForensicSnapshot",
  • "States": {
  • "CreateVolumeForensicSnapshot": {
  • "Type": "Task",
  • "Resource": "arn:aws:states:::aws-sdk:ec2:createSnapshot",
  • "Parameters": {
  • "VolumeId.$": "$.detail.volumeId",
  • "Description": "Forensic preservation snapshot"
  • },
  • "End": true
  • }
  • }
  • }
  • ],
  • "Next": "AwaitSecurityLeadTerminationApproval"
  • }

Both branches execute simultaneously. The state machine pauses until both branches successfully complete before transitioning to the next state.

Human-in-the-Loop Orchestration with Task Tokens

Fully autonomous remediation carries risk: accidentally terminating a critical database primary or revoking access for an executive during business hours can cause severe disruptions. Task Tokens ( .waitForTaskToken ) enable safe human-in-the-loop approvals.

Task Token Lifecycle

Pausing Execution : In a Task state, Step Functions appends .waitForTaskToken to the resource ARN and injects a unique token ( $$.Task.Token ) into the task parameters: {

  • "Type": "Task",
  • "Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
  • "Parameters": {
  • "QueueUrl": "https://sqs.us-east-1.amazonaws.com/123456789012/IncidentApprovalQueue",
  • "MessageBody": {
  • "IncidentId.$": "$.incidentId",
  • "InstanceId.$": "$.detail.instanceId",
  • "TaskToken.$": "$$.Task.Token"
  • }
  • },
  • "TimeoutSeconds": 3600,
  • "HeartbeatSeconds": 300,
  • "Next": "ExecuteGracefulTermination"
  • }

Dispatching Approval Request : A worker or Lambda function consumes the SQS message and publishes an actionable alert to Slack, Microsoft Teams, or email containing Approve and Reject buttons referencing the token.

Resuming or Aborting :

If the operator approves: The external system invokes aws stepfunctions send-task-success --task-token <TOKEN> --output '{"status": "APPROVED"}' , resuming execution.

If rejected: The system invokes aws stepfunctions send-task-failure --task-token <TOKEN> --error "ApprovalDenied" --cause "User rejected termination" , triggering catch/abort branches.

If TimeoutSeconds expires before a token callback: Step Functions automatically fails the task with States.Timeout .

AWS Systems Manager Incident Manager

Important

Current availability: AWS Systems Manager Incident Manager stopped accepting new customers on November 7, 2025, and AWS states that it will receive no new features. Existing customers can continue using it. New customers should use Systems Manager OpsCenter and Automation with their chosen on-call and collaboration platform.

For existing customers, AWS Systems Manager Incident Manager provides incident response management, escalation, and post-incident analysis alongside custom Step Functions workflows.

Core Incident Manager Primitives

Response Plans : Pre-configured incident response templates. A Response Plan defines who is notified, how the incident is escalated, which chat channels are engaged (Amazon Q Developer in chat applications integration with Slack), and which SSM Automation runbooks execute automatically upon incident initiation.

Contacts & Escalation Plans : Defines individual responders, their contact channels (SMS, voice call, email), and tiered escalation paths. For example, Tier 1 on-call engineer is paged via SMS; if unacknowledged after 15 minutes, Tier 2 secondary engineer receives a voice call.

Engagement Plans : Groups multiple contacts and escalation plans based on incident severity (Critical, High, Medium, Low).

Automated Runbook Execution & Post-Incident Analysis

Upon trigger (via CloudWatch Alarm or EventBridge), Incident Manager:

Automatically executes diagnostic or mitigation SSM Automation runbooks (e.g., gathering application stack traces, checking database connection pool saturation, or rolling back a failed deployment).

Gathers CloudWatch metrics, CloudTrail logs, and alarm states into a centralized Incident Dashboard.

Creates a collaboration channel via Amazon Q Developer in chat applications in Slack.

Generates a Post-Incident Analysis (PIR) report documenting the incident timeline, metrics, Time to Detect (TTD), Time to Acknowledge (TTA), and Time to Resolve (TTR), generating actionable Jira or SSM OpsCenter remediation tasks.