Study guide
9.3 Accelerating Data Models & TSIDX Acceleration
Splunk Core Certified Power User Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 Exam Facts, Logistics & Test-Taking Strategy 0.2 Splunk Knowledge Objects & Search Pipeline Architecture
2 Chapter 1: Transforming Commands for Visualizations
1.1 The chart Command & Multi-Dimensional Aggregations 1.2 The timechart Command & Time-Series Visualizations 1.3 Transforming Options: limit, useother, usenull, and Split-By Clauses
3 Chapter 2: Filtering and Formatting Results
2.1 The eval Command & Core Evaluation Functions 2.2 Filtering Data: search vs. where Commands 2.3 Handling Missing Data with fillnull
4 Chapter 3: Correlating Events & Transactions
3.1 Understanding Transactions & Event Grouping Concepts 3.2 Configuring transaction Constraints: maxspan, maxpause, startswith, endswith 3.3 Analyzing Transaction Output Fields: duration & eventcount 3.4 transaction vs. stats: Performance & Architectural Trade-offs
5 Chapter 4: Creating and Managing Fields
4.1 Regular Expression Extractions with Field Extractor 4.2 Delimiter-Based Extractions with Field Extractor 4.3 Field Extraction Management, Permissions & Search-Time Precedence
6 Chapter 5: Field Aliases and Calculated Fields
5.1 Creating and Using Field Aliases 5.2 Creating and Configuring Calculated Fields 5.3 Aliases vs. Calculated Fields in the Search Execution Pipeline
7 Chapter 6: Tags and Event Types
6.1 Creating and Using Event Types 6.2 Creating and Managing Tags 6.3 Combining Tags and Event Types for Data Classification
8 Chapter 7: Creating and Using Search Macros
7.1 Creating Basic Search Macros & Backtick Syntax 7.2 Macro Arguments, Validation & Evaluation Functions 7.3 Macro Scoping, Permissions & Troubleshooting
9 Chapter 8: Creating and Using Workflow Actions
8.1 Workflow Action Architecture & Field/Event Context 8.2 Creating GET and POST Workflow Actions 8.3 Creating Search Workflow Actions & Dynamic Secondary Searches
10 Chapter 9: Data Models and Pivot
9.1 Data Model Architecture & The Pivot Tool 9.2 Creating and Structuring Datasets & Hierarchies 9.3 Accelerating Data Models & TSIDX Acceleration
11 Chapter 10: Common Information Model (CIM)
10.1 The Splunk Common Information Model (CIM) & Normalization 10.2 CIM Knowledge Objects, Data Models & Schemas 10.3 Normalizing & Validating Data with the CIM App / datamodel command
9.3 Accelerating Data Models & TSIDX Acceleration
Key Takeaways Explain This Topic AI
Data Model Acceleration (DMA) creates high-performance columnar time-series index summary files (.tsidx) directly on indexers, speeding up searches across billions of events from minutes to sub-second responses.
Only Root Event datasets and their descendant child datasets can be accelerated; Root Search datasets and Root Transaction datasets are strictly ineligible for acceleration.
Summary ranges (1 day, 7 days, 1 month, 3 months, 1 year, all time) define the historical time window for which TSIDX acceleration summaries are built and maintained on disk.
Accelerated data models are queried using the high-performance | tstats and | pivot commands, which bypass raw journal decompression to read directly from .tsidx columnar index files.
Using summariesonly=t in | tstats queries guarantees that search execution strictly reads from pre-built TSIDX summary files, maximizing query velocity for production dashboards.
Last updated: August 2026
9.3 Accelerating Data Models & TSIDX Acceleration
Quick Summary: Data Model Acceleration (DMA) is an enterprise-grade performance optimization feature in Splunk that creates lightweight, columnar Time-Series Index ( .tsidx ) summary files directly on indexers. Rather than scanning raw index buckets, decompressing raw event journals ( journal.zst / journal.gz ), and parsing search-time field extractions during every query, accelerated searches read directly from pre-computed .tsidx summary files. This reduces query execution times from minutes or hours to sub-second responses. DMA powers Splunk Enterprise Security (ES), ITSI, executive dashboards, and high-speed | tstats search pipelines.
1. The Mechanics of Data Model Acceleration & TSIDX Architecture
To appreciate why Data Model Acceleration is essential in large-scale deployments, consider what occurs during a standard, unaccelerated Splunk search over high-volume data (e.g., querying 500 million web access logs over a 30-day window): +------------------------------------------------------------------------------------------------+
- | UNACCELERATED SEARCH VS. ACCELERATED TSIDX SEARCH |
- | |
- | 1. UNACCELERATED RAW SEARCH PIPELINE (Slow / High CPU & Disk I/O): |
- | +-------------+ +-------------------+ +------------------+ +-------------+ |
- | | Match Index | ---> | Decompress Raw | ---> | Execute Search- | ---> | Pipeline | |
- | | Time Buckets| | Journal Files | | Time Extractions | | Transforming| |
- | | (Metadata) | | (Heavy Disk Read) | | (Heavy Regex CPU)| | (stats/eval)| |
- | +-------------+ +-------------------+ +------------------+ +-------------+ |
- | Result: 500 Million Events Processed | Duration: ~180 Seconds |
- | |
- | 2. ACCELERATED DATA MODEL SEARCH PIPELINE (Sub-Second / Pure TSIDX Read): |
- | +------------------------------------------------------------------+ +-------------+ |
- | | Read Columnar Data Directly from Pre-Built .tsidx Summary Files | ---> | Stream | |
- | | (Bypasses journal.zst, regex extractions, and raw event parsing) | | Aggregation | |
- | +------------------------------------------------------------------+ +-------------+ |
- | Result: 500 Million Events Summarized | Duration: ~0.4 Seconds |
- +------------------------------------------------------------------------------------------------+
- How Acceleration Works Under the Hood
When Data Model Acceleration is enabled on a data model:
Background Acceleration Jobs: The Splunk scheduler on the search head initiates automated background summary-generation searches (running by default every 5 minutes) across the indexer cluster.
TSIDX Summary Generation on Indexers: Each indexer processes the raw events residing in its local bucket storage that match the root event dataset constraints. It extracts the defined data model attributes and compiles them into specialized Time-Series Index ( .tsidx ) files.
Parallel Storage in Index Buckets: The generated .tsidx files are stored locally on each indexer directly within the bucket directory structure: $SPLUNK_DB/<index_name>/datamodel_summary/<data_model_id>/<bucket_id>/
Columnar Key-Value Architecture: Unlike raw events that store unstructured text, .tsidx summary files store structured, columnar key-value index maps. When an accelerated search is executed, indexers read only the specific requested columns (e.g., status , bytes , clientip ) directly from the .tsidx files without touching the raw journal files.
2. Acceleration Eligibility & Strict Restrictions
Not all datasets or configurations in Splunk can be accelerated. Splunk enforces strict eligibility criteria: +------------------------------------------------------------------------------------------------+
- | ACCELERATION ELIGIBILITY MATRIX |
- | |
- | ELIGIBLE FOR ACCELERATION: |
- | ✔ Root Event Datasets (and all descendant child/grandchild event datasets). |
- | ✔ Datasets using streaming eval expressions and standard search-time field extractions. |
- | ✔ Datasets enriched with automatic lookups. |
- | |
- | STRICTLY INELIGIBLE FOR ACCELERATION: |
- | ✘ Root Search Datasets (Arbitrary SPL searches with transforming commands cannot build TSIDX).|
- | ✘ Root Transaction Datasets (Multi-event state tracking cannot be indexed into TSIDX). |
- | ✘ Datasets utilizing non-streaming commands or event-order-dependent logic. |
- +------------------------------------------------------------------------------------------------+
- Why Search and Transaction Datasets Cannot Be Accelerated
Root Search Datasets allow arbitrary SPL commands (such as dedup , sort , eventstats , join , or subsearches). These operations require stateful memory tracking across events and cannot be pre-computed into static time-series index blocks.
Root Transaction Datasets group events dynamically across sliding time windows ( maxspan , maxpause ). Because incoming late events can modify past transaction boundaries, static index summary files cannot represent transaction states deterministically.
Acceleration Settings in datamodels.conf
In distributed environments, data model acceleration parameters are configured in datamodels.conf : # $SPLUNK_HOME/etc/apps/<app_name>/local/datamodels.conf
- [Web_Intelligence]
- acceleration = true
- acceleration.earliest_time = -1mon
- acceleration.backfill_time = -1mon
- acceleration.max_time = 3600
- acceleration.schedule_priority = default
Configuration Parameter Description & Valid Values
acceleration = true | false Toggles TSIDX summary acceleration on or off for the data model.
acceleration.earliest_time Defines the historical summary range window (e.g., -1d , -7d , -1mon , -1y , 0 for all time).
acceleration.backfill_time Specifies how far back historical summaries should be built when acceleration is first activated.
acceleration.max_time Maximum execution time in seconds for individual background summary creation searches (defaults to 3600).
acceleration.manual_rebuilds If set to true , disables automatic rebuilding when model schemas change.
3. Configuring Summary Ranges & Backfill Lifecycle
When enabling acceleration in Splunk Web under Settings > Data models > Edit > Edit Acceleration , the administrator must select a Summary Range . +------------------------------------------------------------------------------------------------+
- | ACCELERATION SUMMARY RANGES |
- | |
- | [ ] 1 Day (-1d) --> Minimum storage footprint; ideal for daily operational triage. |
- | [ ] 7 Days (-7d) --> Common default; ideal for weekly KPI tracking & SOC operations. |
- | [*] 1 Month (-1mon) --> Standard enterprise setting; covers monthly reporting cycles. |
- | [ ] 3 Months (-3mon) --> Quarterly audit & compliance reporting. |
- | [ ] 1 Year (-1y) --> Long-term historical trend analysis (requires substantial storage). |
- | [ ] All Time (0) --> Covers entire index retention; highest disk space consumption. |
- +------------------------------------------------------------------------------------------------+
- The Backfill Lifecycle
When acceleration is first enabled on a data model:
Initial Backfill: Splunk starts a series of background searches to build summaries from the present time back to the configured earliest_time . During backfill, the Build Progress percentage increases in Splunk Web (e.g., 25% -> 60% -> 100%).
Ongoing Maintenance: Once backfill reaches 100%, small summary creation jobs run every 5 minutes to index newly arrived events into .tsidx summaries.
Bucket Aging & Pruning: Data model summary files are tightly coupled to the underlying index buckets. When raw index buckets age out and are rolled to frozen/deleted based on index retention policies, the corresponding .tsidx summary files are automatically deleted from disk.
Schema Modification Rebuilds: If an administrator modifies the constraints or attributes of an accelerated data model, Splunk detects that existing .tsidx files no longer match the schema definition. Splunk automatically invalidates the entire summary cache and initiates a 0% to 100% full backfill rebuild from scratch.
4. Accelerated Search Execution: | tstats and | pivot
Accelerated data models provide the data foundation for two primary search mechanisms: the graphical Pivot interface and the high-performance | tstats command.
Anatomy of an Accelerated | tstats Query
The tstats (Table Statistics) command is a generating command that reads directly from .tsidx index files. When querying an accelerated data model, tstats uses the from datamodel clause: | tstats count, sum(Web.bytes) AS total_bytes
- FROM datamodel=Web_Intelligence.Web_Requests
- WHERE Web.status=500
BY Web.clientip, Web.host +------------------------------------------------------------------------------------------------+
- | ANATOMY OF A TSTATS QUERY |
- | |
- | | tstats count, avg(Web.response_time) AS avg_resp |
- | └── 1. STATISTICAL AGGREGATIONS (count, sum, avg, max, min, dc) |
- | |
- | FROM datamodel=Web_Intelligence.Web_Requests |
- | └── 2. TARGET ACCELERATED DATA MODEL & DATASET (<model_name>.<dataset_name>) |
- | |
- | WHERE Web.status >= 400 |
- | └── 3. FILTERING CONSTRAINTS (Prefixed with Model Name) |
- | |
- | BY Web.host, Web.uri_path |
- | └── 4. GROUPING / SPLIT-BY CLAUSE (Prefixed with Model Name) |
- +------------------------------------------------------------------------------------------------+
- The Critical summariesonly Argument
The summariesonly argument in tstats controls fallback behavior when summary files are incomplete or the search time range extends beyond the acceleration summary window:
summariesonly Setting Search Execution Behavior When to Use
summariesonly=false (or f / omitted) Hybrid Search: Queries .tsidx summaries for the accelerated time range, and automatically falls back to scanning raw unaccelerated events for any remaining time range. Interactive exploratory searches where completeness is prioritized over raw speed.
summariesonly=true (or t ) Strict Accelerated Search: Queries exclusively from .tsidx summary files. Any unaccelerated time range or events not yet summarized return 0 results. Production dashboards, Enterprise Security correlation searches, and scheduled KPI reports requiring guaranteed sub-second response times.
SPL Comparison: Standard Search vs. Accelerated | tstats `-- UNACCELERATED SEARCH: 45 seconds over 50M events --`
- index=web sourcetype=access_combined status=500
- | stats count, avg(response_time) AS avg_resp BY host
- `-- ACCELERATED TSTATS SEARCH: 0.2 seconds over 50M events --`
- | tstats summariesonly=t count, avg(Web.response_time) AS avg_resp
- FROM datamodel=Web.Web_Errors
- WHERE Web.status=500
- BY Web.host
- 5. Storage Footprint, Maintenance & Performance Troubleshooting
While Data Model Acceleration delivers dramatic speed improvements, administrators must actively monitor storage overhead and system resources. +------------------------------------------------------------------------------------------------+
- | DMA STORAGE & SYSTEM IMPACT |
- | |
- | INDEXER STORAGE OVERHEAD: |
- | • Typical TSIDX storage overhead: 1% to 5% of raw uncompressed index volume. |
- | • High-cardinality attributes (e.g., unique URLs, hashes, session tokens) increase size. |
- | |
- | INDEXER CPU & DISK I/O OVERHEAD: |
- | • Background summary jobs run every 5 minutes. |
- | • Initial backfill spikes CPU and disk write I/O across indexer clusters. |
- +------------------------------------------------------------------------------------------------+
- Monitoring Data Model Acceleration in Splunk Web
Administrators can inspect acceleration status directly under Settings > Data models by expanding the accordion row next to any data model:
Status: Complete (100%), Building (e.g., 42%), or Disabled.
Size on Disk: Total storage footprint consumed by .tsidx summary files across all indexers (e.g., 14.2 GB ).
Summary Range: Configured window (e.g., 1 Month ).
Updated: Timestamp of the most recent summary update search.
Buckets: Number of index buckets currently containing acceleration summaries.
Auditing Acceleration Jobs via Internal Logs
To troubleshoot acceleration failures, verify build frequency, or detect long-running backfill jobs, query the _internal index: `-- Check Data Model Acceleration background search executions --`
- index=_internal sourcetype=scheduler savedsearch_name="_ACCELERATE_DATAMODEL_*"
- | stats count, avg(run_time) AS avg_sec, max(run_time) AS max_sec BY savedsearch_name, status
- Role Permissions for Acceleration
Enabling and managing Data Model Acceleration requires specific administrative capabilities:
accelerate_datamodel : Required to enable, edit, or disable acceleration on a data model.
admin_all_objects : Required to override global acceleration storage limits in limits.conf .
6. Enterprise Best Practices Checklist for Data Model Acceleration
Prune High-Cardinality Fields: Avoid adding high-cardinality string fields (such as complete URLs with session tokens, raw GUIDs, or arbitrary message strings) to accelerated models unless strictly necessary. High cardinality drastically inflates .tsidx file size.
Right-Size Summary Ranges: Do not select "All Time" by default. Match the summary range to operational reporting windows (e.g., 7 days for SOC operations, 1 month for executive dashboards).
Use summariesonly=t in Production Dashboards: Guarantee sub-second dashboard rendering by enforcing summariesonly=t in all | tstats queries.
Schedule Schema Changes During Maintenance Windows: Because editing constraints or attributes triggers an automatic full summary rebuild, perform schema updates during off-peak hours to avoid cluster I/O contention.
Monitor Disk Watermarks: Ensure indexer storage volumes have sufficient capacity to accommodate both raw index buckets and .tsidx summary files without breaching disk freeze thresholds.