Study guide
17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
Key Takeaways Explain This Topic AI
DEST_KEY = queue with FORMAT = nullQueue drops matching events on the heavy forwarder or indexer, and events dropped this way are not indexed or counted against the license.
Setting queue = nullQueue does not stop later transforms in the chain: a following transform can set queue = indexQueue, which is how drop-everything-except patterns work.
DEST_KEY = MetaData:Index (with the leading underscore) routes events to another index, statically (FORMAT = pci_secure) or from a capture group (FORMAT = tenant $1).
An event routed to an index that does not exist is dropped unless indexes.conf sets lastChanceIndex, which must name an existing, enabled index.
MetaData:Host, MetaData:Source, and MetaData:Sourcetype values must be prefixed with host::, source::, or sourcetype::, for example FORMAT = host::$1.
Last updated: September 2026
Event Routing, Dynamic Index Assignment & Filtering to nullQueue
Quick Summary: Index-time transforms can drop unwanted events, send events to other indexes, and correct metadata. DEST_KEY = queue with FORMAT = nullQueue discards matching events before they are indexed, so they do not count against the license. DEST_KEY = _MetaData:Index changes the index, and MetaData:Host , MetaData:Source , and MetaData:Sourcetype rewrite metadata, provided their values carry the host:: , source:: , or sourcetype:: prefix.
Filtering Noise to nullQueue: Mechanics & License Impact
Not all telemetry generated by enterprise infrastructure provides operational or analytical value. Verbose debug logs, load-balancer health-check pings, and heartbeat packets saturate network queues, consume storage, and inflate licensing costs. Splunk Enterprise provides a built-in mechanism to discard unwanted data at index time: routing to the nullQueue . Typing Queue Routing Logic
- +-----------------------------------+
- | typingQueue |
- | Evaluates transforms.conf Regex |
- +-----------------------------------+
- |
- +----------------+----------------+
- | |
- (Matches Filter Regex) (Does Not Match / Default)
- | |
- v v
- +------------------------+ +------------------------+
- | DEST_KEY = queue | | indexQueue |
- | FORMAT = nullQueue | | (indexed & metered) |
- +------------------------+ +------------------------+
- | |
- v v
- +------------------------+ +------------------------+
- | nullQueue | | Bucket Storage |
- | (Discarded from Memory)| | (Written to Disk) |
- | * ZERO LICENSE COST * | | |
- +------------------------+ +------------------------+
- The Operational Pipeline Mechanism
nullQueue is an internal pipeline sink within the splunkd process. When an event's destination queue is set to nullQueue :
The event enters typingQueue on a Heavy Forwarder or Indexer.
Splunk matches the event against the REGEX in transforms.conf .
DEST_KEY = queue with FORMAT = nullQueue sets the event's queue key to nullQueue . Later transforms in the same chain still run and can change the key again.
After the typing pipeline, an event whose queue is nullQueue is discarded instead of going to indexQueue . It is never written to disk.
The Critical Licensing Advantage
License rule: license usage measures data as it is indexed . Events sent to the nullQueue in the typing pipeline never reach the index pipeline, so they are not counted against the daily license volume. They still cost forwarding bandwidth and parsing CPU, though, because they were read, sent, and parsed first.
This lets you collect a noisy stream, such as debug-level logs or verbose firewall traffic, keep the events that matter, and discard the rest without using license on them.
Concrete nullQueue Configuration: Filtering Kubernetes Health Checks
Suppose an ingress controller logs thousands of HTTP 200 health checks per minute from kube-probe : # $SPLUNK_HOME/etc/apps/<app_name>/local/props.conf
- [kube:ingress:access]
- TRANSFORMS-drop_health = discard_kube_probes
- # $SPLUNK_HOME/etc/apps/<app_name>/local/transforms.conf
- [discard_kube_probes]
- REGEX = (?i)"GET\s+/healthz\s+HTTP/\d\.\d"\s+200\s+.*"kube-probe"
- DEST_KEY = queue
- FORMAT = nullQueue
- Default-Drop vs. Selective-Keep Patterns
In security or compliance use cases, administrators often want to drop everything except a whitelist of critical events. This requires chaining two transforms in props.conf : # props.conf
- [windows:security:filtered]
- TRANSFORMS-01_filter = set_nullqueue_default, keep_critical_event_ids
- # transforms.conf
- # Step 1: Set everything to nullQueue by default
- [set_nullqueue_default]
REGEX = .
- DEST_KEY = queue
- FORMAT = nullQueue
- # Step 2: Rescue specific EventIDs and direct them to indexQueue
- [keep_critical_event_ids]
- REGEX = (?ms)EventCode=(?:4624|4625|4720|4738)\b
- DEST_KEY = queue
- FORMAT = indexQueue
set_nullqueue_default matches every single event ( REGEX = . ) and targets nullQueue .
keep_critical_event_ids runs next, because it comes later in the same comma-separated list . If the event contains EventCode 4624 (successful logon), 4625 (failed logon), 4720 (user account created), or 4738 (user account changed), it sets the queue back to indexQueue .
The order is essential. Reversing the list would send every event to the nullQueue , because the catch-all rule would run last and overwrite the keep decision.
Dynamic Routing to Alternate Indexes
By default, events are written to the index specified in inputs.conf . However, enterprise organizations frequently require routing events to different indexes based on the event's content—for example, separating PCI-scoped transactions from standard operational data, or routing logs by client tenant.
- Static Index Redirection
- To redirect events matching a security signature to a dedicated index: # props.conf
- [custom:app:log]
- TRANSFORMS-pci_route = route_pci_events
- # transforms.conf
- [route_pci_events]
- REGEX = (?i)(?:payment_gateway|pan_authorization|credit_charge)
- DEST_KEY = _MetaData:Index
- FORMAT = pci_secure
DEST_KEY = _MetaData:Index : Note the leading underscore ( _MetaData:Index ). This tells the pipeline engine to update the target index metadata attribute.
FORMAT = pci_secure : Overwrites the destination index with the literal string pci_secure .
Dynamic Index Routing Using Regex Capture Groups
Rather than creating dozens of static stanzas for every application or customer, administrators can capture tokens from the event payload and dynamically generate the destination index name: # props.conf
- [cloud:multitenant:audit]
- TRANSFORMS-dynamic_tenant = route_by_tenant_id
- # transforms.conf
- [route_by_tenant_id]
- REGEX = ^[^{]*\{"tenant":"([a-zA-Z0-9_-]+)"
- DEST_KEY = _MetaData:Index
- FORMAT = tenant_$1
If an incoming event contains {"tenant":"finance", ...} , the capture group $1 extracts finance .
FORMAT = tenant_$1 resolves to tenant_finance .
The event is dynamically routed to the tenant_finance index.
Prerequisite: every index that FORMAT can produce must exist on all indexers (defined in indexes.conf , pushed through the cluster manager in a cluster). Per indexes.conf.spec , an event whose index does not exist, whether from an invalid index in inputs.conf or an invalid _MetaData:Index from FORMAT , is dropped entirely , unless lastChanceIndex names an existing, enabled index to receive it. With lastChanceIndex = default , the defaultDatabase index is used.
Overriding Core Metadata: Host, Sourcetype, and Source
In complex network topologies, default metadata assigned at ingestion is frequently inaccurate. A classic example is a centralized syslog aggregator (e.g., syslog-ng or a Heavy Forwarder) receiving events from hundreds of network switches. By default, Splunk sets host to the IP or hostname of the syslog relay, masking the identity of the true originating device.
Destination Keys for Metadata Overrides
Metadata Target DEST_KEY Syntax FORMAT Syntax Common Administrative Application
Destination Index _MetaData:Index <index_name> or $1 Segregating PCI/HIPAA data or multi-tenant routing.
Originating Host MetaData:Host host::<hostname> or host::$1 (prefix required) Extracting the real host from syslog relayed through another system
Sourcetype MetaData:Sourcetype sourcetype::<name> (prefix required) Reclassifying mixed data arriving on a shared port
- Source MetaData:Source source::<value> (prefix required) Normalizing source values
- Pipeline Queue queue nullQueue or indexQueue Dropping events, or keeping selected ones
Forwarding Group _TCP_ROUTING Comma-separated tcpout group names Sending selected events to particular output groups (on a forwarder)
Syntax: _MetaData:Index has a leading underscore, while MetaData:Host , MetaData:Sourcetype , and MetaData:Source do not. transforms.conf.spec says key names are case-sensitive and must be used exactly as listed, and that the host, source, and source type values must carry their host:: , source:: , or sourcetype:: prefixes.
- Practical Example: Syslog Host Extraction and Sourcetype Reassignment
- Consider a network stream arriving on TCP 514 containing Cisco ASA and Palo Alto firewall logs: # props.conf
- [syslog:network:collector]
- TRANSFORMS-01_host = extract_syslog_origin_host
- TRANSFORMS-02_type = classify_cisco_asa, classify_palo_alto
- # transforms.conf
- [extract_syslog_origin_host]
- REGEX = ^(?:<\d+>)?(?:[A-Z][a-z]{2}\s+\d+\s+[\d:]+)\s+([a-zA-Z0-9\.\-_]+)\s+
- DEST_KEY = MetaData:Host
- FORMAT = host::$1
[classify_cisco_asa]
REGEX = %ASA-\d-\d+:
- DEST_KEY = MetaData:Sourcetype
- FORMAT = sourcetype::cisco:asa
- [classify_palo_alto]
- REGEX = ,TRAFFIC,|,THREAT,|,SYSTEM,
- DEST_KEY = MetaData:Sourcetype
- FORMAT = sourcetype::pan:traffic
- Troubleshooting Transforms & Avoiding Race Conditions
Transform configurations operate at index time, meaning errors affect data permanently. Administrators must be skilled at debugging pipeline routing.
1. Validating Configuration Merging with btool
To verify that props.conf and transforms.conf stanzas have been parsed correctly without typos or stanza overrides from other apps: # Inspect merged transforms definitions and app origin
$SPLUNK_HOME/bin/splunk btool transforms list --debug | grep -A 7 "\[discard_kube_probes\]"
- # Inspect merged props mappings
- $SPLUNK_HOME/bin/splunk btool props list --debug | grep -A 4 "\[kube:ingress:access\]"
- 2. Checking splunkd.log and the Results
A regex that fails to compile, or a transform with missing settings, produces warnings or errors in splunkd.log on the instance that parses the data (heavy forwarder or indexer), not on the search head: index=_internal sourcetype=splunkd (log_level=WARN OR log_level=ERROR) host=<parsing instance>
Then verify the effect with searches. Check that dropped events stop appearing, that routed events land in the new index ( index=pci_secure ), and that the license usage report reflects the reduction.
3. Pipeline Race Conditions and Execution Ordering
When configuring multiple transforms that alter routing and metadata, execution ordering is critical:
Rule order: transforms in a chain all run, and a later transform that sets the same key overwrites an earlier one. In a keep pattern, the catch-all nullQueue rule must come before the keep rule.
Mutual Exclusivity: When multiple rules set DEST_KEY = _MetaData:Index , the last rule executed wins . Ensure regex patterns are mutually exclusive, or use strictly ordered class prefixes ( 01_ , 02_ ) so that broad general routing precedes specific override rules.
nullQueue is decided at the end: setting queue = nullQueue does not immediately remove the event. Later transforms still see it and can set indexQueue again. Only the final value when the typing pipeline finishes decides whether the event is dropped.