ExamFinch.com My Study

Study guide

11.2 Transform Processor In Action

Practice this exam

OpenTelemetry Certified Associate OTCA

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Exam Overview & Observability Foundations

1.1 OTCA Exam Structure & Observability Paradigms 1.2 Core Telemetry Signals: Traces, Metrics, Logs & Baggage 1.3 Telemetry Analysis, SLOs & High Cardinality

2 Chapter 2: Semantic Conventions & Instrumentation Strategy

2.1 OpenTelemetry Semantic Conventions & Namespaces 2.2 Resource Attributes & Entity Modeling 2.3 Zero-Code vs Code-Based Instrumentation

3 Chapter 3: OpenTelemetry Architecture & The Data Model

3.1 Architectural Separation of API and SDK 3.2 The Core Trace Model: Spans, SpanContext & Links 3.3 Span Events, Attributes & Status Codes 3.4 Composability & Extension: Plugin Interfaces, Distributions & Contrib

4 Chapter 4: Distributed Tracing & Context Propagation

4.1 In-Process & Cross-Process Context Propagation 4.2 W3C Trace Context Specification 4.3 W3C Baggage & Alternative Propagators

5 Chapter 5: OpenTelemetry Metrics API & Instruments

5.1 Synchronous Metric Instruments 5.2 Asynchronous (Observable) Metric Instruments 5.3 Metric Aggregations & Temporality

6 Chapter 6: Logs Data Model & Logging Bridge API

6.1 OpenTelemetry Logs Data Model & Architecture 6.2 The Logging Bridge API & Log Appenders

7 Chapter 7: SDK Pipelines, Processors & Exporters

7.1 Span Processors & Exporters 7.2 Metric Readers & Log Processors 7.3 Standard Exporters & OTLP Wire Protocols

8 Chapter 8: Sampling Strategies, SDK Configuration & Agents

8.1 Head-Based Sampling & Built-in Samplers 8.2 Tail-Based Sampling vs Head-Based Sampling 8.3 OpenTelemetry SDK Configuration & Environment Variables 8.4 Zero-Code Agents: Configuration, Kubernetes Injection & Troubleshooting

9 Chapter 9: OpenTelemetry Collector Architecture & Pipelines

9.1 Collector Core Architecture & Component Lifecycle 9.2 Collector Configuration Structure & Pipelines 9.3 Essential Collector Extensions

10 Chapter 10: Collector Components: Receivers, Processors & Exporters

10.1 Standard Receivers & Ingestion Protocols 10.2 Core Processors & Execution Order 10.3 Standard Exporters & Delivery Guarantees

11 Chapter 11: Data Transformation & The OpenTelemetry Transformation Language (OTTL)

11.1 OTTL Concepts & Syntax Grammar 11.2 Transform Processor In Action 11.3 Advanced Connectors

12 Chapter 12: Collector Deployment, Scaling & Resilience

12.1 Agent vs Gateway Deployment Topologies 12.2 Scaling, Load Balancing & High Availability 12.3 Resilience: Backpressure, Buffering & Persistent Queues

13 Chapter 13: Maintaining, Debugging & Schema Evolution

13.1 Debugging Collector Pipelines & Telemetry Health 13.2 Troubleshooting Broken Traces & Context Loss 13.3 Schema Management & Telemetry Evolution

11.2 Transform Processor In Action

Key Takeaways Explain This Topic AI

The transform processor (available in the OpenTelemetry Collector Contrib distribution) is the primary engine executing OTTL statements across traces, metrics, and logs pipelines.

Transform processor configuration organizes statements by signal and context blocks (trace_statements, metric_statements, log_statements), executing each list sequentially.

OTTL editors such as set, replace_pattern, replace_all_patterns, delete_key, keep_keys, and merge_maps change telemetry, while converters such as ParseJSON, ExtractPatterns, Concat, IsMatch, and Int return values for arguments and conditions.

The ParseJSON() function parses stringified JSON payloads into structured maps, enabling extraction of nested properties into queryable OpenTelemetry attributes.

The error_mode setting controls behavior on statement errors: 'ignore' (the default: log and continue), 'silent' (continue without logging), and 'propagate' (return the error, which drops the payload).

Last updated: September 2026

11.2 Transform Processor In Action

Quick Answer: The transform processor (part of the OpenTelemetry Collector Contrib distribution) is the execution engine that applies OTTL transformation statements to telemetry streams. Configured inside the processors: pipeline section, it organizes transformations into signal-specific lists ( trace_statements , metric_statements , log_statements ). In the basic style each list holds plain statements with context-qualified paths (for example set(span.attributes["x"], "y") ), and the processor infers the context; the advanced style shown below groups statements under an explicit context (such as span , datapoint , or log ) with optional conditions and a per-group error_mode . Key functions include the editors replace_pattern (PII redaction), set (attribute injection and normalization), and delete_key / keep_keys (sanitization), and the converter ParseJSON (turning stringified JSON bodies into maps).

In enterprise environments, the Collector acts as the central data gateway. Before forwarding data to multiple backend destinations (such as Datadog, Prometheus, Elasticsearch, or AWS CloudWatch), telemetry must often be transformed to meet strict compliance mandates, strip sensitive data, align with organizational naming conventions, or populate missing fields. The transform processor provides the computational power to perform these operations in-stream without needing external code or sidecars.

Architecture and Configuration of the Transform Processor

The transform processor processes batches of telemetry records as they flow through the pipeline. It parses each OTTL statement at startup, compiles it into internal Go abstract syntax trees, and executes the statements sequentially on each record.

Core YAML Configuration Structure processors:

transform:

error_mode: ignore # ignore | silent | propagate

trace_statements:

- context: span

statements:

- set(attributes["processed.by"], "otel-collector")

metric_statements:

- context: datapoint

statements:

- set(attributes["cluster"], "us-east-k8s") where metric.name == "system.cpu.load"

log_statements:

- context: log

statements:

  • set(severity_number, 17) where IsMatch(body, "(?i)error|exception")
  • The error_mode Directive

When evaluating statements over millions of unpredictable user payloads, occasional runtime errors occur (e.g., regex parsing failures, invalid type conversions, or accessing non-existent map elements). The error_mode setting dictates how the processor handles these conditions:

ignore (the default when error_mode is not set, and the recommended mode): Logs the error and continues with the next statement. Telemetry is preserved.

silent : Suppresses error logging completely and proceeds to the next statement. Used in high-volume production where expected missing fields would otherwise flood collector logs.

propagate : Returns the error up the pipeline, which drops the payload from the Collector. Use it when strict compliance requires that data which failed a redaction step is never exported.

Essential OTTL Function Catalog

OTTL functions come in two kinds. Editors (lowercase names) change telemetry and begin every statement. Converters (UpperCamelCase names) return a value that you pass to an editor or test in a where condition.

1. Editors: Changing Telemetry

set(target, value) : Writes value to target , creating the attribute if it does not exist and overwriting it if it does.

replace_pattern(target, regex, replacement) : Replaces every match of regex inside the string field target . The replacement can reference capture groups ( $1 ); inside a Collector configuration file each $ must be written as $$ so that environment-variable substitution leaves it alone.

replace_all_patterns(target, mode, regex, replacement) : Applies the same replacement to every value ( mode "value" ) or every key ( mode "key" ) of a map such as attributes .

delete_key(target, key) : Deletes key from the map target (e.g., delete_key(attributes, "authorization.token") ); a missing key is a no-op.

keep_keys(target, keys) : Drops every key in the map except those listed, implementing strict allow-listing.

merge_maps(target, source, strategy) : Merges a map (often the result of a converter) into target using insert , update , or upsert .

2. Converters: Strings

Concat(values[], delimiter) : Joins values with a delimiter (e.g., Concat([attributes["env"], attributes["service"]], "-") ).

Split(target, delimiter) : Splits a string into a list.

Substring(target, start, length) : Returns part of a string (byte offsets).

ToLowerCase(target) / ToUpperCase(target) : Change case.

3. Converters: Parsing and Extraction

ParseJSON(target) : Parses a JSON string into a map; index it directly ( ParseJSON(body)["user_id"] ) or merge it with merge_maps .

ExtractPatterns(target, pattern) : Returns a map of the named capture groups in pattern ; combine it with merge_maps(attributes, ExtractPatterns(body, "..."), "upsert") .

4. Converters: Types and Predicates

Int(val) / Double(val) / String(val) / Bool(val) : Convert values between types.

IsMatch(target, regex) : Returns true when the string matches the regex; used in where conditions.

HasPrefix(target, prefix) / HasSuffix(target, suffix) : Fast string checks that return Boolean values.

  • Real-World OTTL Use Cases
  • Use Case 1: Masking PII & Sensitive Headers (PCI-DSS & GDPR)

Observability pipelines must prevent personally identifiable information (PII) and secret credentials from leaking into telemetry backends. processors:

transform/pii_redaction:

error_mode: ignore

trace_statements:

- context: span

statements:

# 1. Mask credit card numbers (13-16 digits) with [REDACTED_PCI]

- replace_pattern(attributes["http.request.body"], "\\b(?:\\d[ -]*?){13,16}\\b", "[REDACTED_PCI]") where attributes["http.request.body"] != nil

# 2. Scrub Bearer authentication tokens from HTTP authorization headers

- replace_pattern(attributes["http.request.header.authorization"], "Bearer [A-Za-z0-9\\-\\._~\\+/]+=*", "Bearer [REDACTED_TOKEN]") where attributes["http.request.header.authorization"] != nil

  • # 3. Completely delete user social security numbers from span attributes
  • delete_key(attributes, "user.ssn")
  • Use Case 2: Normalizing Legacy Attributes to Modern Semantic Conventions

When microservices transition across OpenTelemetry SDK versions, attribute names often change (for instance, older HTTP conventions used http.status_code , whereas newer semantic conventions mandate http.response.status_code ). processors:

transform/normalize_semantics:

error_mode: ignore

trace_statements:

- context: span

statements:

# Migrate http.status_code to http.response.status_code if modern key is absent

- set(attributes["http.response.status_code"], attributes["http.status_code"]) where attributes["http.response.status_code"] == nil and attributes["http.status_code"] != nil

  • # Delete legacy key once migrated
  • delete_key(attributes, "http.status_code") where attributes["http.response.status_code"] != nil
  • # Ensure response status code is an Integer

- set(attributes["http.response.status_code"], Int(attributes["http.response.status_code"]) ) where attributes["http.response.status_code"] != nil

Use Case 3: Parsing Structured Fields from Unstructured Log Bodies

Legacy application frameworks frequently output JSON-formatted logs as raw text strings directly into the body field. Storing the body as a raw string makes searching and dashboarding difficult. OTTL can unpack the JSON string into top-level attributes: processors:

transform/json_logs:

error_mode: ignore

log_statements:

- context: log

statements:

  • # Extract user_id and ip_address from JSON log body
  • # ($$ is how a regex end anchor is written inside a Collector config file)
  • set(attributes["user.id"], ParseJSON(body)["user_id"]) where IsMatch(body, "^\\{.*\\}$$")
  • set(attributes["client.ip"], ParseJSON(body)["client_ip"]) where IsMatch(body, "^\\{.*\\}$$")
  • # Extract transaction amount and cast to Float
  • set(attributes["transaction.amount"], Double(ParseJSON(body)["amount"])) where IsMatch(body, "^\\{.*\\}$$")
  • Use Case 4: Dynamic Log Severity Assignment

Many legacy applications output all log entries with severity_number: 0 (Unspecified) or default to INFO (severity 9) even when reporting fatal exceptions. OTTL can inspect the message content and assign the appropriate OpenTelemetry severity number: processors:

transform/log_severity:

error_mode: ignore

log_statements:

- context: log

statements:

  • # If body contains FATAL or CRITICAL, elevate severity to FATAL (21)
  • set(severity_number, 21) where IsMatch(body, "(?i)fatal|critical")
  • set(severity_text, "FATAL") where severity_number == 21
  • # If body contains ERROR or EXCEPTION, elevate severity to ERROR (17)
  • set(severity_number, 17) where IsMatch(body, "(?i)error|exception") and severity_number < 17
  • set(severity_text, "ERROR") where severity_number == 17
  • Production Pipeline Integration: End-to-End Collector Config

The following configuration demonstrates how the transform processor coordinates with the batch processor inside the collector pipeline: receivers:

otlp:

protocols:

  • grpc: { endpoint: "0.0.0.0:4317" }
  • http: { endpoint: "0.0.0.0:4318" }

processors:

batch:

  • send_batch_size: 1024
  • timeout: 1s

transform/cleanup:

error_mode: ignore

trace_statements:

- context: span

statements:

- replace_pattern(attributes["db.query.text"], "(?i)password=\\S+", "password=[REDACTED]")

log_statements:

- context: log

statements:

- set(attributes["log.extracted_user"], ParseJSON(body)["user"]) where IsMatch(body, "^\\{.*\\}$$")

exporters:

otlp:

  • endpoint: "telemetry-backend:4317"
  • tls: { insecure: true }

service:

pipelines:

traces:

  • receivers: [otlp]
  • processors: [transform/cleanup, batch]
  • exporters: [otlp]

logs:

  • receivers: [otlp]
  • processors: [transform/cleanup, batch]
  • exporters: [otlp]