Study guide
17.1 Transformation Mechanics: props.conf & transforms.conf
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
17.1 Transformation Mechanics: props.conf & transforms.conf
Key Takeaways Explain This Topic AI
props.conf selects the data and invokes TRANSFORMS-<class> = <stanza>,...; transforms.conf holds REGEX, FORMAT, DEST_KEY, and related settings.
Index-time TRANSFORMS run in the typing pipeline on heavy forwarders or indexers; a universal forwarder ignores them unless the UF-only force_local_processing = true is set for that source type.
Transforms listed in one TRANSFORMS-<class> value run in list order, and each one sees the event as changed by the previous one.
At index time FORMAT defaults to <stanza-name>::$1, DEST_KEY is required unless WRITE_META = true, and LOOKAHEAD (default 4096 characters) limits how far into the event REGEX searches.
Metadata keys need prefixes in FORMAT: MetaData:Host values start with host::, MetaData:Source with source::, and MetaData:Sourcetype with sourcetype::.
Last updated: September 2026
Transformation Mechanics: props.conf & transforms.conf
Quick Summary: Index-time transformations are split across two files. props.conf chooses which data is affected and invokes transforms with TRANSFORMS-<class> , and transforms.conf defines what each transform does ( REGEX , FORMAT , DEST_KEY ). They run in the typing pipeline, so they take effect on heavy forwarders and indexers. A universal forwarder ignores them unless force_local_processing is enabled on it.
Architectural Division of Labor: props.conf vs. transforms.conf
Splunk Enterprise manages data onboarding and index-time transformations through a two-file configuration model. This architectural separation decouples where and when an action occurs from what and how the data is manipulated. +-----------------------------------------------------------------------+
- | props.conf |
- | Identifies Input Scope: [<spec>] (sourcetype, source, or host) |
- | Invokes Transform Class: TRANSFORMS-<class> = <stanza1>, <stanza2> |
- +-----------------------------------------------------------------------+
- |
- v (Calls Stanza Definition)
- +-----------------------------------------------------------------------+
- | transforms.conf |
- | Defines Mutation Logic: [<stanza1>] |
- | Directives: REGEX, FORMAT, DEST_KEY, WRITE_META, LOOKAHEAD |
- +-----------------------------------------------------------------------+
- 1. The Role of props.conf (Scoping & Orchestration)
In props.conf , administrators identify the context of the incoming data stream. The stanza header defines the matching boundary:
[<sourcetype>] : Applies the rule to all data assigned the specified sourcetype (most common and recommended practice).
[source::<source>] : Applies the rule to events matching a source pattern (using glob wildcards, e.g., [source::.../access*.log] ).
[host::<host>] : Applies the rule to data arriving from a specific host.
Within that stanza, the TRANSFORMS-<class> directive references one or more transformation definitions located in transforms.conf : # $SPLUNK_HOME/etc/apps/<app_name>/local/props.conf
- [custom:app:json]
- TRANSFORMS-route_and_filter = drop_healthcheck, route_critical_to_sec
Here, <class> represents a unique label ( route_and_filter ) used to differentiate multiple transform directives. It points directly to stanza headers declared in transforms.conf .
2. The Role of transforms.conf (Mutation Specification)
In transforms.conf , each stanza header matches a name referenced by props.conf . This file contains the granular regular expression matching, field formatting, and destination queue or metadata targeting instructions.
Parameter Purpose & Permissible Values Default Behavior
REGEX Regular expression pattern (Perl Compatible Regular Expression - PCRE). Capturing groups () extract sub-patterns for use in FORMAT . None (Required for regex-based transforms)
FORMAT The value written to DEST_KEY . Uses $1 , $2 , … for capture groups; $0 is the DEST_KEY value before the transform ran At index time: <stanza-name>::$1
DEST_KEY Where the FORMAT result is stored: queue , _raw , _MetaData:Index , MetaData:Host , MetaData:Source , MetaData:Sourcetype , _TCP_ROUTING , _SYSLOG_ROUTING (case-sensitive) No default; required for index-time transforms unless WRITE_META = true
- WRITE_META When true , the result is written to the event's metadata as an indexed field false
- LOOKAHEAD How many characters into the event REGEX searches (all index-time transforms) 4096
REPEAT_MATCH Index-time only: run REGEX repeatedly, each time starting where the last match stopped (ignored when DEST_KEY = _raw ) false
- DEFAULT_VALUE Index-time only: value written to DEST_KEY if REGEX fails empty
- SOURCE_KEY The key the REGEX is applied to _raw
- Processing Tiers: Where Transformations Execute
Index-time TRANSFORMS run in the typing pipeline , in the regex-replacement processor. The pipelines in order are parsing (line breaking), merging (timestamps and line merging), typing ( TRANSFORMS , SEDCMD ), and index. Universal forwarder (default)
- input -> (character encoding, input-time settings) -> tcpout to 9997
- * no line breaker, aggregator, or regex-replacement processor
- * TRANSFORMS in its props.conf have no effect
- |
- v (unparsed data)
- Heavy forwarder or indexer
- parsingQueue -> line breaking (LINE_BREAKER, TRUNCATE)
- aggQueue -> timestamps, line merging
- typingQueue -> TRANSFORMS / SEDCMD (regex replacement)
- queue = nullQueue -> event dropped
- _MetaData:Index -> index changed
- indexQueue -> written to the index (or forwarded, on a heavy forwarder)
- The Tier Rule
Universal forwarders: by default, a UF does not break lines, merge lines, or run regex replacement, so TRANSFORMS-<class> in its props.conf does nothing, and the data leaves unmodified. The exception is the UF-only props.conf setting force_local_processing = true . For that source type, it makes the forwarder run the line breaker, aggregator, and regex-replacement processors locally, at the cost of extra CPU and memory.
Heavy forwarders: a full Splunk instance parses data and applies TRANSFORMS before forwarding. Masking, filtering, host overrides, and routing done there reach the indexers already applied.
Indexers: apply TRANSFORMS to data that arrives unparsed, for example from universal forwarders. props.conf.spec notes that the processor ignores TRANSFORMS on an indexer if they were already processed on the heavy forwarder . The newer RULESET-<class> setting (ingest actions) differs: it is processed on both tiers.
Transformation Execution Sequence and Chaining
Enterprise architectures frequently require applying several discrete transformations to a single event stream—for example, scrubbing an API key, overriding the host header, and routing high-priority errors to a dedicated index. Splunk provides two distinct mechanisms for chaining multiple transforms:
Chaining Method 1: Comma-Separated List in a Single Directive
Administrators can list multiple transform stanzas separated by commas under a single TRANSFORMS-<class> key in props.conf : # props.conf
- [cisco:asa]
- TRANSFORMS-pipeline = strip_syslog_header, override_asa_host, route_security_index
Evaluation Rule: Splunk executes comma-separated transforms strictly in left-to-right order :
strip_syslog_header executes first.
override_asa_host executes second, operating on the event state produced by the first transform.
route_security_index executes third, evaluating against the modified payload and metadata.
Chaining Method 2: Multiple Directives via Class Suffixes
Alternatively, administrators can declare multiple TRANSFORMS keys, each with a unique <class> identifier: # props.conf
- [cisco:asa]
- TRANSFORMS-01_strip_header = strip_syslog_header
- TRANSFORMS-02_override_host = override_asa_host
- TRANSFORMS-03_route_index = route_security_index
Evaluation rule: when a stanza has several TRANSFORMS-<class> settings, they are applied in ASCII (lexicographic) order of the <class> names , not in the order they are written in the file:
01_strip_header executes before 02_override_host .
02_override_host executes before 03_route_index .
Best practice: when order matters, put the transforms in one comma-separated list, which props.conf.spec guarantees is applied in list order. If you use separate classes, give them sortable names ( 01_ , 02_ , …). Otherwise TRANSFORMS-clean runs before TRANSFORMS-route simply because c sorts before r .
Sequential Event State Mutation
Transformations are not isolated evaluations; they mutate the event in memory as it traverses the typingQueue . Consider an event entering the pipeline: Raw Ingest: "2026-09-23 10:14:02 gw01.corp ASA-6-302013: Built inbound TCP connection..."
Step 1 ( strip_syslog_header ): Modifies _raw by removing the timestamp and relay prefix. The event in memory is now: "ASA-6-302013: Built inbound TCP connection..."
Step 2 ( override_asa_host ): sets DEST_KEY = MetaData:Host with FORMAT = host::$1 . The value must carry the host:: prefix. Because Step 1 already removed the host from _raw , this transform must capture it before Step 1 runs, or read it from another key with SOURCE_KEY . Order matters.
Step 3 ( route_security_index ): Evaluates REGEX = ^ASA-6- against the current state of _raw . Because the syslog header was already stripped in Step 1, the regex matches immediately at index 0 ( ^ ). If Step 1 had not executed, an unanchored regex would have been required.
Advanced transforms.conf Attributes: LOOKAHEAD & WRITE_META
Beyond basic regex matching, fine-tuning transformation behavior requires mastery of operational attributes:
1. The LOOKAHEAD Setting
By default, LOOKAHEAD = 4096 , so REGEX searches only the first 4,096 characters of the event.
If an event is a large stack trace or a 32 KB XML or JSON payload, and the text needed for routing or masking appears at character 8,000, the transform does not match .
- In such scenarios, administrators must explicitly raise LOOKAHEAD in transforms.conf : [route_large_payloads]
- REGEX = "audit_classification":\s*"RESTRICTED"
- DEST_KEY = _MetaData:Index
- FORMAT = compliance_vault
- LOOKAHEAD = 32768
- 2. WRITE_META and Indexed Fields
Setting WRITE_META = true writes the FORMAT result (for example FORMAT = err_code::$1 ) into the event's metadata as an indexed field , stored in the index files alongside the other indexed terms. The spec recommends WRITE_META = true rather than DEST_KEY = _meta .
Search Impact: Allows searches like fieldname::value to be resolved entirely at the indexer storage layer.
Cost: every indexed field adds to the index files and to indexing work. Splunk's general guidance is to prefer search-time extractions and to create indexed fields only where they bring a clear benefit, such as frequently used fields in very large data sets.
Administrative Traps and Troubleshooting
Failure Scenario Underlying Administrative Trap Diagnostic Technique & Resolution
Transform does not trigger on a universal forwarder props.conf and transforms.conf placed on a UF, which does not run the typing pipeline by default Put them on the heavy forwarder or indexer tier (or, rarely, use force_local_processing on the UF)
Transform fails on long events Regex target is located past byte 4,096, exceeding the default LOOKAHEAD . Increase LOOKAHEAD in transforms.conf (e.g., LOOKAHEAD = 16384 ).
Chained rules run in the wrong order Separate TRANSFORMS-<class> settings run in ASCII order of class names Use one comma-separated list, or rename the classes so they sort correctly
Host or source override has no effect FORMAT value missing its host:: , source:: , or sourcetype:: prefix Use for example FORMAT = host::$1
Transform ignored on the indexer The data was already parsed by a heavy forwarder Put the transform on the heavy forwarder
Transform modifies search-time instead of index-time Using REPORT-<class> instead of TRANSFORMS-<class> . REPORT operates strictly on Search Heads at search time. Use TRANSFORMS-<class> for index-time pipeline operations.