Study guide

12.2 Sensitive Data Masking in Telemetry & Messaging

AWS Certified Security – Specialty SCS-C03

Track your progress as you read

Finished sections get a check mark.

1 Chapter 0: Getting Started with the SCS-C03 Exam

0.1 SCS-C03 Exam Format, Domains & Study Strategy

2 Chapter 1: Security Monitoring, Alerting & Threat Detection Services

1.1 Workload Monitoring & Health Check Strategies 1.2 Amazon GuardDuty Architecture & Finding Types 1.3 AWS Security Hub & Automated Conformance Assessment 1.4 Amazon Macie & Automated Data Classification

3 Chapter 2: Enterprise Logging Architecture & Analysis

2.1 Multi-Account CloudTrail & Organization Trails 2.2 Amazon Security Lake & OCSF Log Normalization 2.3 Network Telemetry: VPC Flow Logs & Route 53 Resolver Logs 2.4 Troubleshooting Security Telemetry & Log Ingestion 2.5 Analyzing, Normalizing & Correlating Security Logs

4 Chapter 3: Incident Readiness, Response Planning & Automated Runbooks

3.1 Incident Response Planning & Blast Radius Mitigation 3.2 Testing & Validating Incident Response Plans 3.3 Automated Incident Remediation with EventBridge & Step Functions 3.4 Systems Manager OpsCenter & Incident Manager Runbooks

5 Chapter 4: Digital Forensics, Evidence Preservation & Root Cause Analysis

4.1 Forensic Artifact Capture & Chain of Custody 4.2 Threat Containment & Network Quarantine Techniques 4.3 Correlating Events & Root Cause Analysis with Amazon Detective

6 Chapter 5: Edge Protection, WAF & DDoS Mitigation

5.1 CloudFront Security Headers & Origin Access Control 5.2 AWS WAF Architecture & Advanced Rule Sets 5.3 AWS Shield Advanced & DDoS Defense Architectures 5.4 Edge Integrations: Managed Rule Groups, OCSF Ingestion, IoT Policies & CORS

7 Chapter 6: Compute Workload Hardening, Vulnerability Management & GenAI Security

6.1 Golden AMIs & Container Image Hardening Pipelines 6.2 Amazon Inspector Vulnerability Management & Patch Automation 6.3 Generative AI Security: Amazon Bedrock Guardrails & Model Protection 6.4 Instance Profiles, Service Roles & Execution Roles for Compute 6.5 Secure Administrative Access: Session Manager & EC2 Instance Connect

8 Chapter 7: Network Security, Segmentation, Private Connectivity & Encryption in Transit

7.1 Layered Traffic Filtering: SGs, NACLs & AWS Network Firewall 7.2 Hybrid Connectivity Security: VPN, Direct Connect & MACsec 7.3 Zero-Trust Access & Network Access Analysis 7.4 Enforcing Encryption in Transit: ELB Security Policies, TLS & HTTPS-Only Access 7.5 Private Access: VPC Endpoints, PrivateLink, Client VPN & Verified Access 7.6 Inter-Resource Encryption in Transit: Nitro, EMR, EKS & SageMaker AI

9 Chapter 8: Enterprise Authentication, Federation & Directory Services

8.1 AWS IAM Identity Center & Enterprise IdP Integration 8.2 Amazon Cognito User Pools & Identity Pools 8.3 AWS Security Token Service (STS) & Temporary Credential Workflows 8.4 Troubleshooting Identity & Authentication Failures

10 Chapter 9: Advanced Authorization Architecture & Policy Evaluation

9.1 Complete AWS Policy Evaluation Logic & Decision Flow 9.2 Permission Boundaries & Safe Admin Delegation 9.3 Attribute-Based Access Control (ABAC) with IAM Tags

11 Chapter 10: Least Privilege, Policy Validation & External Access Auditing

10.1 IAM Roles Anywhere & Amazon Verified Permissions 10.2 Auditing External & Unused Access with IAM Access Analyzer 10.3 Analyzing Authorization Failures & Policy Simulation

12 Chapter 11: Cryptographic Services: KMS Architecture & Key Governance

11.1 AWS KMS Architecture, Key Policies & Key Types 11.2 KMS Grants, Encryption Context & Multi-Region Keys 11.3 Imported Key Material (BYOK) & CloudHSM Key Stores

13 Chapter 12: Secrets Management, Sensitive Data Masking & PKI

12.1 AWS Secrets Manager & Systems Manager Parameter Store 12.2 Sensitive Data Masking in Telemetry & Messaging 12.3 AWS Certificate Manager (ACM) & Private CA Architecture

14 Chapter 13: Storage Encryption, Integrity, Lifecycle & Backup Protection

13.1 S3 Encryption Modes & S3 Bucket Keys 13.2 Data Integrity Protection: Object Lock, Vault Lock & Code Signing 13.3 Secure Data Lifecycle, Cross-Account Replication & Backup Vault Lock 13.4 Lifecycle, Backup & Secure Replication Beyond S3: EFS, FSx, DLM, DataSync & Air-Gapped Vaults

15 Chapter 14: Multi-Account Architecture, Organizations & Control Tower Governance

14.1 AWS Organizations Structure, Hierarchy & Resource Control 14.2 AWS Control Tower Landing Zone & Guardrail Framework 14.3 Delegated Administrator Architecture & Root User Governance 14.4 Centralized Resource Sharing with RAM & Service Catalog

16 Chapter 15: Compliance Automation, Infrastructure as Code Security & Auditing

15.1 Infrastructure as Code Security & Multi-Account Deployment 15.2 AWS Config, Conformance Packs & Automated Remediation 15.3 Audit Evidence Collection & Well-Architected Security Reviews

12.2 Sensitive Data Masking in Telemetry & Messaging

Key Takeaways Explain This Topic AI

CloudWatch Logs data protection detects sensitive data when events are ingested and masks it at every egress point, including the console, Logs Insights, metric filters, and subscription filters; events ingested before the policy stay unmasked.

Data protection policies use managed data identifiers (pattern matching and machine learning, with checksums such as Luhn for card numbers) plus up to 10 custom data identifiers, each a name and a regex of up to 200 characters.

Viewing unmasked log data requires the logs:Unmask IAM permission; users without it, including broad read-only roles, see asterisks in place of each match.

Amazon SNS message data protection (closed to new customers on April 30, 2026) can audit, de-identify (mask or redact), or deny sensitive data on publish (inbound) or delivery (outbound).

Sensitive data audit findings from both CloudWatch Logs and SNS can be routed to Amazon S3 compliance buckets, CloudWatch log groups, or Amazon Data Firehose to maintain verifiable audit trails for PCI DSS, HIPAA, and GDPR compliance.

Last updated: September 2026

12.2 Sensitive Data Masking in Telemetry & Messaging

Software applications routinely generate massive volumes of diagnostic logs, operational traces, and asynchronous event messages. During application debugging, error handling, or high-throughput transaction processing, developers and automated systems frequently log raw request payloads, stack traces, and database queries. Without strict guardrails, these telemetry streams inadvertently capture sensitive Personally Identifiable Information (PII) —such as Social Security numbers (SSNs), credit card primary account numbers (PANs), passport numbers, driver's license details, and cryptographic API tokens.

Exposing sensitive data within logging repositories and messaging brokers introduces severe compliance violations under regulatory standards such as PCI DSS Requirement 3 (protecting cardholder data), HIPAA (safeguarding electronic protected health information), and GDPR/CCPA (protecting consumer privacy). To neutralize these threats without burdening application developers with complex sanitization logic, AWS provides native, inline data protection capabilities: Amazon CloudWatch Logs Data Protection Policies and Amazon SNS Message Data Protection .

CloudWatch Logs Data Protection Architecture

Amazon CloudWatch Logs data protection policies audit and mask sensitive data in log groups. Detection happens as each event is ingested ; masking is applied at every egress point (console, GetLogEvents , FilterLogEvents , Logs Insights, metric filters, and subscription filters). CloudWatch Logs keeps the original event, which is why a principal with logs:Unmask can still view it. You can attach one policy to a log group or create an account-level policy ( PutAccountPolicy ) that covers every existing and future log group in the account; when both exist, identifiers from both apply.

Loading diagram...

Operational Mechanics: Auditing vs. Masking

A CloudWatch Logs data protection policy supports two distinct operational modes within its policy statements:

Audit ( Audit ) :

Scans incoming log events and detects sensitive data patterns without modifying the underlying log text.

Generates structured audit metrics and emits comprehensive finding logs to designated destinations (such as a separate security log group, an Amazon S3 compliance bucket, or an Amazon Data Firehose delivery stream).

Allows security teams to quantify PII exposure and establish baseline metrics prior to enforcing active redaction.

De-identify / Mask ( Deidentify ) :

Replaces each detected value with asterisks whenever the data leaves CloudWatch Logs.

Example: A log line containing User SSN: 123-45-6789 is shown to a user without logs:Unmask with the whole SSN replaced by asterisks; the digits are not partially revealed.

Logs Insights queries, metric filters, and subscription filters also receive masked values, so downstream consumers process sanitized data. Masking applies only to events ingested after the policy exists.

Managed Data Identifiers vs. Custom Data Identifiers

The data protection policy engine detects sensitive data using two mechanisms:

1. Managed Data Identifiers

AWS provides pre-configured Managed Data Identifiers , which combine pattern matching and machine learning (some also require nearby keywords), that detect standard sensitive data formats across multiple geographic jurisdictions and compliance frameworks:

Financial Identifiers : Credit card numbers (validated using the Luhn checksum algorithm to eliminate false positives), American Express, Diners Club, Discover, JCB, MasterCard, Visa, US bank routing numbers, International Bank Account Numbers (IBAN).

Personal Identifiers (PII) : US Social Security numbers (SSNs, validating format, valid area numbers, and range exclusions), US individual taxpayer identification numbers (ITIN), passport numbers (US, UK, Germany, Canada), driver's license numbers, postal addresses, email addresses, phone numbers.

National Identifiers : UK National Insurance Numbers (NINO), Canadian Social Insurance Numbers (SIN), French INSEE, German National Identity Cards.

Security Credentials : AWS Access Key IDs ( AKIA... ), AWS Secret Access Keys, SSH private keys, PGP private keys, JSON Web Tokens (JWT).

2. Custom Data Identifiers

Organizations frequently handle proprietary or domain-specific sensitive data that standard managed identifiers cannot classify—such as internal employee badge IDs, patient medical record numbers (MRNs), or custom loyalty card numbers. CloudWatch Logs supports Custom Data Identifiers :

Name and Regex : Each custom identifier is a name (up to 128 characters) and a regular expression (up to 200 characters) matching the exact format of the proprietary data, for example EMP-[0-9]{6}-[A-Z]{2} . Omit ^ and $ anchors, because the value usually sits in the middle of a log line.

Limits : Up to 10 custom identifiers per policy, defined in the policy's Configuration block and referenced by name. Unlike Amazon Macie, CloudWatch Logs custom identifiers have no keyword or proximity settings, so write specific patterns to limit false positives.

Data Protection Policy JSON Structure

The following policy audits findings to an S3 security bucket and masks the same identifiers, including one custom identifier. The Audit and Deidentify statements must list exactly the same data identifiers: {

  • "Name": "payment-service-data-protection",
  • "Description": "Audit and mask card numbers, SSNs, secret keys, and employee IDs",
  • "Version": "2021-06-01",
  • "Configuration": {
  • "CustomDataIdentifier": [
  • { "Name": "EmployeeId", "Regex": "EMP-[0-9]{6}-[A-Z]{2}" }
  • ]
  • },
  • "Statement": [
  • {
  • "Sid": "AuditSensitiveTelemetry",
  • "DataIdentifier": [
  • "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber",
  • "arn:aws:dataprotection::aws:data-identifier/Ssn-US",
  • "arn:aws:dataprotection::aws:data-identifier/AwsSecretKey",
  • "EmployeeId"
  • ],
  • "Operation": {
  • "Audit": {
  • "FindingsDestination": {
  • "S3": { "Bucket": "centralized-security-compliance-audit-logs" }
  • }
  • }
  • }
  • },
  • {
  • "Sid": "DeidentifySensitiveTelemetry",
  • "DataIdentifier": [
  • "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber",
  • "arn:aws:dataprotection::aws:data-identifier/Ssn-US",
  • "arn:aws:dataprotection::aws:data-identifier/AwsSecretKey",
  • "EmployeeId"
  • ],
  • "Operation": {
  • "Deidentify": { "MaskConfig": {} }
  • }
  • }
  • ]
  • }
  • De-anonymization & Unmask Permissions: logs:Unmask

A central design principle of CloudWatch Logs data protection is least-privilege visibility . When data protection is enabled on a log group:

Standard users, developers, and security analysts possessing standard permissions ( logs:FilterLogEvents , logs:GetLogEvents , logs:StartQuery ) only see masked data .

The CloudWatch Logs console displays a prominent banner indicating that data protection is active and that sensitive terms have been redacted.

The logs:Unmask IAM Permission

In authorized operational scenarios—such as a fraud investigation, security incident triage, or regulatory audit—a privileged engineer may require access to the raw, unmasked data. Viewing unmasked data requires the explicit IAM permission logs:Unmask . {

  • "Version": "2012-10-17",
  • "Statement": [
  • {
  • "Sid": "AllowPrivilegedUnmaskWithMFA",
  • "Effect": "Allow",
  • "Action": "logs:Unmask",
  • "Resource": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/production/payment-service:*",
  • "Condition": {
  • "Bool": {
  • "aws:MultiFactorAuthPresent": "true"
  • }
  • }
  • }
  • ]
  • }
  • Operational Unmasking Mechanics

Console Experience : A user with logs:Unmask can click an "Unmask" toggle button within the CloudWatch Logs console. The console prompts the user to confirm temporary unmasking. Once confirmed, the raw data appears in the current session.

API / CLI Experience : GetLogEvents , FilterLogEvents , and GetLogRecord accept an unmask parameter ( --unmask in the CLI). The call succeeds with original values only if the caller has logs:Unmask ; Logs Insights queries use the unmask(@message) function with the same permission.

CloudTrail Auditability : Every invocation of logs:Unmask is explicitly logged in AWS CloudTrail . The CloudTrail record captures the exact IAM principal, timestamp, source IP address, target log group, and session context. This non-repudiation mechanism ensures that any attempt to view unmasked PII generates an immutable audit trail for security compliance.

Amazon SNS Message Data Protection

Important

Amazon SNS message data protection is no longer available to new customers (since April 30, 2026). Existing users can keep using it; new designs can inspect payloads in the publisher, or send messages through Lambda or Amazon Comprehend PII detection before publishing.

While CloudWatch Logs protects telemetry, Amazon Simple Notification Service (SNS) provides pub/sub messaging across distributed microservices, serverless architectures, and third-party webhook endpoints. When an upstream payment service publishes transaction notifications, message payloads may contain credit card numbers or customer addresses.

Amazon SNS Message Data Protection scans message payloads in real time as they are published to an SNS topic, evaluating the payload against a topic Data Protection Policy .

Loading diagram...

Enforcement Actions: Block vs. De-identify

An SNS data protection policy can execute two distinct operational actions upon detecting sensitive data:

Block ( Deny ) :

Publisher Rejection : With DataDirection: Inbound , the publish request is rejected and the publisher receives an AuthorizationError . With DataDirection: Outbound , delivery to the matching subscriptions is blocked instead.

Zero Ingestion : The message is dropped completely and never delivered to any topic subscribers. This provides an ironclad boundary against sensitive data ingress.

De-identify / Redact ( Deidentify ) :

Inline Payload Transformation : The message is accepted by the SNS topic, but the sensitive data strings are automatically masked or redacted before the message is fanned out to downstream subscribers.

Subscriber Protection : Downstream subscribers (such as Amazon SQS queues, AWS Lambda functions, Amazon Kinesis streams, HTTPS webhooks, or email recipients) receive only sanitized payloads.

SNS Data Protection Policy Example

The following policy configures an Amazon SNS topic to block any message containing Social Security numbers or AWS secret keys from being published, while auditing detections to Amazon CloudWatch Logs: {

  • "Name": "payment-topic-data-protection",
  • "Description": "Block SSNs and AWS secret keys; audit card numbers",
  • "Version": "2021-06-01",
  • "Statement": [
  • {
  • "Sid": "BlockCredentialsAndSSN",
  • "DataDirection": "Inbound",
  • "Principal": ["*"],
  • "DataIdentifier": [
  • "arn:aws:dataprotection::aws:data-identifier/Ssn-US",
  • "arn:aws:dataprotection::aws:data-identifier/AwsSecretKey"
  • ],
  • "Operation": { "Deny": {} }
  • },
  • {
  • "Sid": "AuditFinancialData",
  • "DataDirection": "Inbound",
  • "Principal": ["*"],
  • "DataIdentifier": [
  • "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber"
  • ],
  • "Operation": {
  • "Audit": {
  • "SampleRate": "99",
  • "FindingsDestination": {
  • "CloudWatchLogs": { "LogGroup": "/aws/vendedlogs/sns/payment-topic-audit" }
  • }
  • }
  • }
  • }
  • ]
  • }
  • Comparison: CloudWatch Logs vs. Amazon SNS Data Protection

Capability CloudWatch Logs Data Protection Amazon SNS Message Data Protection

Operational Boundary Telemetry ingestion into log groups. Real-time message publication to topics.

Supported Actions Audit and Deidentify (Masking). Audit , Deidentify (Redaction/Masking), and Deny (Block).

Blocking / Ingress Denial No (detects at ingestion and masks on read; never rejects PutLogEvents ). Yes (Rejects sns:Publish call with HTTP 403 error).

De-anonymization Mechanism Yes : Privileged users with logs:Unmask can unmask. No : Once redacted, downstream subscribers cannot unmask.

Managed Identifiers Full suite (Financial, PII, Credentials, National IDs). Full suite (Financial, PII, Credentials, National IDs).

Custom Regex Identifiers Supported (name and regex, up to 10 per policy). Supported (name and regex).

Audit Findings Destinations Amazon S3, CloudWatch Logs, Amazon Data Firehose. Amazon S3, CloudWatch Logs ( /aws/vendedlogs/ log group), Amazon Data Firehose.

Availability Generally available. Closed to new customers since April 30, 2026.

Specialty Exam Pitfalls & Architectural Traps

The Retrospective Masking Fallacy : Believing that enabling a CloudWatch Logs data protection policy will retroactively mask pre-existing log events in the log group. Sensitive data is detected as events are ingested , so events ingested before the policy was set are never masked. To reduce exposure in historical logs, restrict read access to the log group, shorten retention, or delete old log streams after exporting what compliance requires.

The AdministratorAccess Unmask Assumption : Assuming that an IAM principal with the AWS managed policy AdministratorAccess can view unmasked logs by default. While AdministratorAccess grants * on * , if an explicit Deny or Permissions Boundary restricts logs:Unmask , or if an analyst views the log group without initiating an unmask session, the data remains masked. Always verify whether an analyst possesses explicit, positive authorization for logs:Unmask .

SNS Block Breaking Critical Pipelines : Configuring an SNS message data protection policy with a Deny (Block) action on high-priority operational topics. If an application encounters an unexpected error and includes an API key or email address in a critical alert message, the SNS topic will reject the message entirely, dropping the alert and blinding operations teams to an active outage. Use Deidentify rather than Deny when message delivery must be maintained.

Overly Broad Custom Regex : Defining custom data identifier patterns that are too generic (CloudWatch Logs custom identifiers have no keyword or proximity option to narrow them). Unanchored regular expressions frequently trigger massive false positive rates across standard diagnostic telemetry, resulting in unintended masking of harmless hexadecimal identifiers, hash values, and system UUIDs.

Loading diagram... CloudWatch Logs and SNS Data Protection Enforcement Pipeline