Study guide
15.1 Input-Phase Processing & Default Metadata Assignment
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
15.1 Input-Phase Processing & Default Metadata Assignment
Key Takeaways Explain This Topic AI
The input assigns host, source, sourcetype, and index from inputs.conf before parsing; index defaults to the default index (main) and host to the [default] host name.
host_segment = N uses the Nth /-separated path segment; on Windows the drive letter does not count, so D:\logs\servers\host01 with host_segment = 3 gives host01.
host_regex uses the first capture group matched against the source value, and if host_regex and host_segment are both set, host_regex is ignored.
connection_host defaults to dns for TCP inputs and ip for UDP inputs; none keeps the host configured in inputs.conf.
In props.conf, [source::] settings override [host::], which override [<sourcetype>]; setting sourcetype explicitly avoids automatic classification and generic default parsing.
Last updated: September 2026
Input-Phase Processing & Default Metadata Assignment
Quick Summary: Every event carries four default metadata fields: host , source , sourcetype , and index . They are assigned by the input that reads the data, on the forwarder or on the first full Splunk instance, before parsing starts. You control them in inputs.conf with static values, path-based host extraction ( host_segment , host_regex ), and connection_host for network inputs. Setting them explicitly, especially sourcetype , is what makes the later parsing rules in props.conf apply correctly.
Where Metadata Is Assigned in the Pipeline
Splunk's data pipeline starts with the input segment, followed by the parsing , merging , typing , and index pipelines (covered in detail in the staging chapter):
Input: the input reads the data (file, network port, script, Windows API), breaks it into blocks, and attaches host , source , sourcetype , and index from inputs.conf . Character encoding ( CHARSET ) is also applied here.
Parsing pipeline: line breaking ( LINE_BREAKER ) and header processing.
Merging pipeline: timestamp extraction and, when SHOULD_LINEMERGE = true , line merging.
Typing pipeline: regex-based TRANSFORMS and SEDCMD , such as masking, routing, and metadata rewrites.
Index pipeline: writing raw data and index files to buckets, or forwarding.
Why Input-Time Metadata Matters
props.conf matching: parsing settings are found by matching the event's metadata against props.conf stanzas. When several stanzas match, [source::<source>] overrides [host::<host>] , which overrides [<sourcetype>] . rule:: and delayedrule:: stanzas are only used to classify data that has no source type. If the input assigns the wrong source type, the right parsing rules never run.
Routing and filtering: transforms that send events to other indexes, output groups, or the nullQueue usually key on host , source , or sourcetype .
Index placement: the index setting decides where data lands and therefore which roles can search it and which retention settings apply. It defaults to the default index ( main ).
- Host Assignment in inputs.conf
- 1. Static Host ( host = <value> )
Every event from the stanza gets that host value. If a stanza does not set host , the value comes from the [default] stanza of inputs.conf , which Splunk sets to the machine's host name at installation. [monitor:///opt/oracle/admin/diag/rdbms/prod/alert.log]
- host = db-cluster-primary.corp.example
- sourcetype = oracle:alert:log
- index = databases
- 2. Host from a Path Segment ( host_segment = <N> )
When a central server stores logs in per-host directories (for example, a syslog server writing /var/log/remote_hosts/<host>/... ), host_segment uses the Nth / -separated segment of the path as the host.
Path: /var/log/remote_hosts/web-prod-01/nginx/access.log
- Segment Value
- 1 var
- 2 log
- 3 remote_hosts
- 4 web-prod-01
- 5 nginx
- 6 access.log [monitor:///var/log/remote_hosts/.../*.log]
- host_segment = 4
- sourcetype = nginx:access
- index = web_logs
If the value is not an integer or is less than 1, the default host setting is used.
Windows: the drive letter and colon do not count as a segment. With host_segment = 3 and the path D:\logs\servers\host01 , the host is host01 .
3. Host from a Regular Expression ( host_regex = <regex> )
When host names sit at varying depths or inside file names, host_regex applies a regex to the path and uses the first capturing group as the host. [monitor:///mnt/storage/.../logs/*.log]
host_regex = /mnt/storage/(?:prod|dr)/([^/]+)/
- [monitor:///var/log/syslog-*.log]
- host_regex = syslog-[a-z]+-([a-zA-Z0-9-]+)\.corp\.log
The regex is matched against the source value. If source is set explicitly in the stanza, that string is matched instead of the file name.
If the regex does not match, the default host setting is used.
Which Setting Wins?
Per inputs.conf.spec : if host_regex and host_segment are both set, host_regex is ignored. If the chosen method does not produce a value, the stanza's host , or the [default] host, is used. A source type with a host-override transform (as with syslog data) can still replace host later in the typing pipeline.
4. Network Inputs ( connection_host )
For [tcp://] and [udp://] inputs, the host usually comes from the sender:
- Value Behavior Default for
- dns Reverse DNS name of the sender's IP; forward and reverse DNS should match TCP inputs
- ip The sender's IP address, with no lookup UDP inputs
- none Keep the host set in inputs.conf , typically the receiving instance's host name –
For [splunktcp] inputs, the forwarder normally sends its own host value. connection_host is used only if the remote instance does not set a host. When data passes through a relay, the sender's address is the relay's, so extract the true host from the event (for example, the syslog header) with a host-override transform.
Source Type Assignment
The source type decides how Splunk breaks events, finds timestamps, and applies field extractions.
Explicit vs. Automatic
Explicit ( sourcetype = <name> ) : you name the source type in the input. This is the best practice for every production input.
Automatic : if the input sets no source type, Splunk tries to recognize the data. It uses its pretrained source types and the rule:: and delayedrule:: stanzas in props.conf , and as a last resort it generates a new source type based on the source. Very small files can end up as too_small or <sourcename>-too_small .
Why Automatic Source Typing Is Avoided in Production
Inconsistent names: similar files can be classified differently depending on their content, and learned names such as app-too_small or generated source types do not match what add-ons, dashboards, and data models expect.
Generic parsing: a source type with no specific settings runs on the [default] values in props.conf , such as SHOULD_LINEMERGE = true and MAX_TIMESTAMP_LOOKAHEAD = 128 . These defaults are slower and less accurate than a tuned source type with LINE_BREAKER , SHOULD_LINEMERGE = false , TIME_PREFIX , and TIME_FORMAT .
Knowledge objects: event types, tags, field aliases, and CIM data models are usually written for specific source type names such as cisco:asa or pan:traffic .
Best practice: set sourcetype explicitly in every input stanza, preferably using the names that the relevant Splunkbase add-on expects.
Source Assignment and Overrides
By default, source is:
Monitored files: the full path of the file, such as /var/log/apache2/access.log .
Network inputs: the protocol and port, such as udp:514 or tcp:1514 .
Scripted and modular inputs: the script path or input name.
source = <name> in the stanza overrides it. This is rarely necessary for files, because the path is valuable information, but it can help when paths contain random container IDs. Remember that host_regex matches the overridden value.