Study guide
16.1 Line Breaking & Event Segmentation
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
16.1 Line Breaking & Event Segmentation
Key Takeaways Explain This Topic AI
Line breaking happens in the parsing pipeline on the indexer or heavy forwarder, not on a universal forwarder, so LINE_BREAKER settings belong on the parsing tier.
LINE_BREAKER needs a capturing group: the start of the first group ends the previous event, the end of the group starts the next event, and the group's text is discarded.
The default LINE_BREAKER is ([\r\n]+) and the default SHOULD_LINEMERGE is true; Splunk recommends SHOULD_LINEMERGE = false with an explicit LINE_BREAKER for speed.
With SHOULD_LINEMERGE = true, the merging pipeline (aggQueue) recombines lines using BREAK_ONLY_BEFORE_DATE (default true), BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, and MAX_EVENTS (default 256 lines).
TRUNCATE (default 10000 bytes) caps line length; set it higher for large JSON events, and treat TRUNCATE = 0 carefully because very long lines often indicate garbage data.
Last updated: September 2026
Line Breaking & Event Segmentation
Quick Summary: Line breaking turns a continuous stream of data into separate events. The recommended configuration is SHOULD_LINEMERGE = false with an explicit LINE_BREAKER regex in props.conf : the regex's first capturing group marks the boundary and is discarded. The alternative, SHOULD_LINEMERGE = true (the default), splits data into lines and then merges them back in the merging pipeline using rules such as BREAK_ONLY_BEFORE_DATE , which is slower. TRUNCATE and MAX_EVENTS limit event size.
Mechanics of Event Segmentation in the Pipeline
Data passes through the input segment and then the parsing , merging , typing , and index pipelines. Event boundaries are created in two steps:
Parsing pipeline ( parsingQueue ) – line breaking: the LineBreakingProcessor splits the incoming stream into initial events wherever LINE_BREAKER matches. TRUNCATE is enforced here.
Merging pipeline ( aggQueue ) – line merging: if SHOULD_LINEMERGE = true , the aggregator ( AggregatorMiningProcessor in splunkd.log ) recombines those initial events into multi-line events. It uses rules such as BREAK_ONLY_BEFORE_DATE and MAX_EVENTS . Timestamp extraction also happens in this pipeline.
Where This Happens
A universal forwarder does not parse data (except for structured-data settings such as INDEXED_EXTRACTIONS ). It sends blocks of data, and line breaking happens on the indexer or heavy forwarder that receives them. props.conf line-breaking settings therefore belong on that parsing tier.
An event can span two blocks of data. LINE_BREAKER_LOOKBEHIND (default 100 bytes) sets how far back from the end of a chunk, with the next chunk appended, Splunk applies LINE_BREAKER . Increase it for very large or multi-line events.
Modern Regex Line Breaking: SHOULD_LINEMERGE = false and LINE_BREAKER
Splunk's [default] stanza still uses line merging ( SHOULD_LINEMERGE = true ), but for any data you onboard deliberately, Splunk recommends letting LINE_BREAKER delimit the events.
- The Core Configuration Pattern
- Set these two settings together in props.conf : [my_custom_sourcetype]
- SHOULD_LINEMERGE = false
- LINE_BREAKER = <regex_with_capturing_group_1>
- The Capturing Group 1 Rule
props.conf.spec defines the behavior precisely:
Rule: The regex must contain a capturing group. Wherever it matches, the start of the first capturing group is the end of the previous event, and the end of the first capturing group is the start of the next event. The contents of the first capturing group are discarded and are not in any event. Matched text before the group stays at the end of the previous event, and matched text after it begins the next event.
- Syntax Breakdown with Concrete Examples
- Example 1: Standard Single-Line Logs
- For single-line logs separated by standard line feeds or carriage returns: LINE_BREAKER = ([\r\n]+)
The regex matches one or more carriage return ( \r ) or newline ( \n ) characters.
Because ([\r\n]+) is enclosed in parentheses, it constitutes capturing group 1 .
Splunk discards the newline characters and splits the stream into discrete single-line events.
Example 2: Multiline Application Logs with Timestamp Headers
Consider an application log where an event begins with a date, but errors generate 50-line Java stack traces: 2026-09-23 10:14:02.124 INFO Application initialized
- 2026-09-23 10:14:05.882 ERROR NullPointerException: database pool exhausted
- at com.corp.db.Pool.connect(Pool.java:42)
- at com.corp.app.Main.run(Main.java:118)
2026-09-23 10:14:06.001 INFO Retrying connection...
To break only before lines starting with a new timestamp while keeping the entire stack trace as a single multiline event: LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}
Capturing Group 1: ([\r\n]+) matches the preceding newline and is discarded .
Uncaptured Pattern: \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} matches the date and time of the new event.
Because the timestamp pattern is outside capturing group 1, Splunk does not discard it. The timestamp is preserved at the very beginning of the new event.
Stack trace lines ( at com.corp.db... ) are not followed by a date pattern after the newline, so no break occurs there, and the whole stack trace stays inside the error event.
Why SHOULD_LINEMERGE = false Is Faster
props.conf.spec notes that you get a significant boost to processing speed when LINE_BREAKER delimits multi-line events, compared with using SHOULD_LINEMERGE to reassemble individual lines. With LINE_BREAKER alone, each event is created in one pass. With line merging, the data is first split into lines and each line is then evaluated against the merge rules.
Attribute LINE_BREAKER + SHOULD_LINEMERGE = false SHOULD_LINEMERGE = true (the [default] )
Where events are formed Parsing pipeline ( LineBreakingProcessor ) Split in the parsing pipeline, then recombined in the merging pipeline ( aggQueue )
How One regex decides every boundary Each line is checked against BREAK_ONLY_BEFORE_DATE , BREAK_ONLY_BEFORE , MUST_BREAK_AFTER , and similar rules
- Relevant limits TRUNCATE TRUNCATE per line, plus MAX_EVENTS lines per event
- Performance Fast; recommended by Splunk Slower, especially at high volume
- Legacy Line Merging in props.conf (SHOULD_LINEMERGE = true)
SHOULD_LINEMERGE = true is the [default] setting, so any source type without its own line-breaking configuration uses line merging.
Legacy Parameters and Evaluation Order
When SHOULD_LINEMERGE = true is set, Splunk uses the following parameters in props.conf to evaluate line boundaries:
Parameter Default Evaluation Mechanics
BREAK_ONLY_BEFORE Empty Breaks the accumulated event before any line matching the specified regex.
BREAK_ONLY_BEFORE_DATE true Create a new event only when a new line with a date is encountered (not meaningful with DATETIME_CONFIG = CURRENT or NONE ).
MUST_BREAK_AFTER Empty Forces an event boundary immediately after any line matching the specified regex.
MUST_NOT_BREAK_BEFORE Empty If the current line matches, do not break the last event before this line.
MUST_NOT_BREAK_AFTER Empty If the current line matches, do not break on any later lines until MUST_BREAK_AFTER matches.
Operational Risks of Legacy Merging
Oversized events: if the format changes so that the break rule stops matching, lines keep being added to one event until MAX_EVENTS is reached. splunkd.log then shows AggregatorMiningProcessor messages about breaking the event because the limit was exceeded.
Queue pressure: line merging runs in the merging pipeline ( aggQueue ). If it cannot keep up, aggQueue fills and blocks the queues before it, parsingQueue and the input queues, which eventually slows forwarders sending to this instance.
Event Sizing and Safety Bounds: TRUNCATE and MAX_EVENTS
Two settings cap how large an event can grow.
1. TRUNCATE = <bytes>
Default Value: 10000 bytes (approximately 10 KB).
Function: the default maximum line length in bytes. For multi-byte characters it is rounded down so that it never lands mid-character.
Mechanics: longer lines are truncated, and splunkd.log records LineBreakingProcessor warnings about truncating lines because the limit was exceeded. When LINE_BREAKER delimits whole events, each event counts as a "line".
The Data Corruption Hazard on Stack Traces and JSON
In enterprise systems, large data structures routinely exceed 10,000 bytes:
Structured JSON Logs: Cloud service logs (such as AWS CloudTrail or Kubernetes audit events) frequently reach 20 KB to 100 KB in size.
Diagnostic Stack Traces: Enterprise application crashes involving deep microservice or database call stacks can span 30 KB.
Warning
Silent JSON Ingestion Failure: If TRUNCATE is left at the default 10000 bytes for JSON sources, Splunk cuts the event in the middle of a string or nested object. The trailing closing brackets ( } ) are severed. Downstream search-time field extractions ( KV_MODE = json or spath ) will fail completely with syntax errors, rendering the event unsearchable by its structured keys.
Best Practice for TRUNCATE
Increase TRUNCATE to accommodate the maximum expected payload size for the sourcetype (for example, TRUNCATE = 100000 or TRUNCATE = 500000 ).
Setting TRUNCATE = 0 means never truncate. The spec warns that very long lines are often a sign of garbage data, so prefer a generous finite value.
2. MAX_EVENTS = <integer>
Default Value: 256 lines.
Function: the maximum number of input lines added to any event during line merging. After that many lines, Splunk breaks the event.
Mechanics: a 400-line stack trace under line merging becomes two events, the first with 256 lines.
Interaction with LINE_BREAKER : with SHOULD_LINEMERGE = false , lines are never merged, so MAX_EVENTS has nothing to limit. The event size is set by where LINE_BREAKER matches and by TRUNCATE .