Study guide
4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle
Key Takeaways Explain This Topic AI
Hot buckets are the only ones being written; warm buckets are closed but stay in homePath; cold buckets are moved to coldPath; all three are searchable.
Hot buckets roll to warm when they reach maxDataSize (auto = 750 MB), span more than maxHotSpanSecs (90 days), exceed maxHotBuckets (auto = 3 per pipeline), sit idle past maxHotIdleSecs, or splunkd restarts.
Warm buckets roll to cold when maxWarmDBCount (default 300) or homePath.maxDataSizeMB is exceeded, oldest first.
Cold buckets freeze when all their events are older than frozenTimePeriodInSecs (default about 6 years) or when maxTotalDataSizeMB (default 500,000 MB) is exceeded; frozen data is deleted unless coldToFrozenDir or coldToFrozenScript is set.
To thaw an archived bucket, copy it into thaweddb, run splunk rebuild on it, and restart; thawed data never ages out and does not count against the license.
Last updated: September 2026
Bucket Types and the Bucket Lifecycle
An index's buckets move through five states, hot, warm, cold, frozen, and thawed , under rules set in indexes.conf . Knowing which setting moves a bucket from one state to the next is one of the most heavily tested index topics.
Bucket Lifecycle Progression
Data moves deterministically through lifecycle tiers based on administrative rules, disk thresholds, and temporal age. [Ingestion Pipeline]
- │
- ▼
- ┌───────────┐
- │ HOT │ ◄── Actively written (Read/Write) in homePath
- └─────┬─────┘
- │ (Trigger: maxDataSize, maxHotSpanSecs, restart, etc.)
- ▼
- ┌───────────┐
- │ WARM │ ◄── Closed, merged .tsidx (Read-Only) in homePath
- └─────┬─────┘
- │ (Trigger: maxWarmDBCount exceeded)
- ▼
- ┌───────────┐
- │ COLD │ ◄── Migrated to coldPath (Read-Only on cheaper storage)
- └─────┬─────┘
- │ (Trigger: frozenTimePeriodInSecs OR maxTotalDataSizeMB hit)
- ▼
- ┌───────────┐ Archive configured? (coldToFrozenDir / coldToFrozenScript)
- │ FROZEN ├───────────────────────────────┬───────────────────────────────┐
- └───────────┘ │ │
- [YES] ▼ [NO] ▼
- ┌───────────────┐ ┌───────────────────┐
- │ Archived to │ │ Permanently │
- │ External Store│ │ Deleted from Disk │
- └───────┬───────┘ └───────────────────┘
- │ (Manual Admin Restore)
- ▼
- ┌───────────────┐
- │ THAWED │ ◄── Restored to thawedPath; rebuilt via
- └───────────────┘ 'splunk rebuild'; exempt from aging
- 1. Hot Tier Mechanics
When events emerge from the indexing pipeline, the index processor writes them into an active hot bucket located in homePath . Hot buckets are named using internal identifiers: hot_v1_<sub_id> (such as hot_v1_0 ).
Hot buckets are the only buckets that permit write operations.
Hot buckets are fully searchable. Splunk handles concurrency so that search head queries can read data from hot buckets simultaneously while new events are being appended.
2. Warm Tier Mechanics
When a hot bucket reaches an operational limit, Splunk closes the bucket and rolls it to warm. During this roll:
Splunk flushes all in-memory buffers to disk.
The bucket is renamed from hot_v1_<sub_id> to standard warm naming syntax: db_<latest_time>_<earliest_time>_<localid>
Small tsidx files continue to be merged by the optimize process.
No further events are written to the bucket.
The warm bucket remains in homePath .
Note
Notice the timestamp sequence in the bucket folder name: db_<latest_time>_<earliest_time>_<localid> . The latest epoch timestamp appears first, followed by the earliest epoch timestamp, followed by a monotonically increasing local bucket ID (e.g., db_1695427200_1695340800_12 ). In an indexer cluster, the originating copy appends the source peer's GUID ( db_<newest>_<oldest>_<localid>_<guid> ), and replicated copies use the rb_ prefix ( rb_<newest>_<oldest>_<localid>_<guid> ).
3. Cold Tier Mechanics
As new warm buckets roll from hot, the number of warm buckets in homePath grows. When the count exceeds maxWarmDBCount (default 300), or homePath exceeds homePath.maxDataSizeMB if you set one, Splunk rolls the oldest warm bucket to coldPath ( $SPLUNK_DB/<index_name>/colddb ).
Cold buckets maintain the exact same internal directory structure and filename as warm buckets.
Cold buckets remain fully searchable.
By separating coldPath from homePath , organizations can retain historical compliance logs on dense, lower-cost storage without degrading performance on the high-speed NVMe storage hosting hot and warm data.
4. Frozen Tier Mechanics
Buckets freeze when every event in the bucket is older than frozenTimePeriodInSecs , or when the index exceeds its size limit ( maxTotalDataSizeMB , which freezes the oldest data first). A frozen bucket is no longer searchable.
Splunk handles frozen buckets in one of two ways:
Archived: If coldToFrozenDir is set, Splunk copies the bucket there, keeping only its rawdata . If coldToFrozenScript is set, Splunk runs your script to archive it.
Deleted: If neither archiving attribute is defined, Splunk permanently erases the bucket files from disk.
5. Thawed Tier Mechanics
Thawed data represents historical events that have been restored from frozen archives to satisfy legal audits, forensic inquiries, or compliance reviews.
Restored buckets are placed in thawedPath ( $SPLUNK_DB/<index_name>/thaweddb ).
Crucial Rule: Thawed buckets are completely exempt from automated aging rules. They will never roll to frozen or be automatically purged by maxTotalDataSizeMB or frozenTimePeriodInSecs . The administrator must manually delete thawed buckets once search investigations conclude.
Hot Bucket Roll Triggers
A hot bucket does not stay open forever. These conditions roll it to warm: # Hot roll settings in indexes.conf (defaults shown)
- maxDataSize = auto # 750 MB (auto_high_volume = 10 GB on 64-bit)
- maxHotSpanSecs = 7776000 # 90 days of event time in one hot bucket
- maxHotBuckets = auto # 3 hot buckets per ingestion pipeline
- maxHotIdleSecs = 0 # 0 = no idle timeout
- 1. maxDataSize Reached
When the size of a hot bucket's rawdata journal and tsidx files exceeds maxDataSize , the bucket rolls to warm. Default settings include:
auto : Sets the maximum size to 750 MB .
auto_high_volume : Sets the maximum size to 10,000 MB (10 GB) on 64-bit operating systems (750 MB on 32-bit systems). Recommended for indexes receiving high daily data volume to prevent bucket proliferation.
Integer value: Can be explicitly set in megabytes (e.g., maxDataSize = 2048 ).
2. maxHotSpanSecs Exceeded
Specifies the maximum time range (in seconds) between the earliest and latest event allowed inside an individual hot bucket. The default value is 7776000 seconds (90 days).
If a hot bucket already contains an event from January 1, and an incoming event arrives timestamped April 5, the time span between the events exceeds 90 days.
Splunk immediately rolls the hot bucket to warm and creates a new hot bucket for the incoming event, preventing buckets from spanning excessive temporal ranges.
3. maxHotBuckets Exceeded
Defines the maximum number of hot buckets per index. The default auto means 3 , applied per ingestion pipeline , so an indexer with N pipeline sets can have up to N × 3 hot buckets for one index.
If an incoming event carries a timestamp that does not fit into any open hot bucket, Splunk must open a new hot bucket.
If opening a new hot bucket would exceed maxHotBuckets , Splunk rolls the hot bucket that contains the least recent data to warm.
Separately, maxHotIdleSecs (default 0 , disabled) rolls a hot bucket that has received no data for that many seconds.
4. Indexer Daemon Restart or Shutdown
Whenever the splunkd service is stopped or restarted (e.g., splunk restart , system reboot, or software upgrade), Splunk cleanly flushes all in-memory buffers to disk and immediately rolls all currently open hot buckets to warm .
Upon restarting, Splunk initializes brand new hot buckets ( hot_v1_0 ) for all active indexes.
Frequent indexer restarts create large numbers of tiny warm buckets, which degrades search performance and wastes inode capacity.
5. Time Clock Skew & Out-of-Order Timestamps
If an endpoint or forwarder sends events with badly skewed timestamps (e.g., a batch of historical logs timestamped two years in the past, or an improperly configured device reporting events in the future), those events may not fit the time span of any open hot bucket ( maxHotSpanSecs ). Splunk opens another hot bucket, which can push an older one out under maxHotBuckets . Badly skewed timestamps therefore cause many small buckets. Values below 3600 for maxHotSpanSecs are reset to 3600.
Restoring Frozen Data & The Thawing Process
When legal or compliance work needs data that has already aged out to a frozen archive, the administrator restores it by thawing the archived bucket.
What an Archived Bucket Contains
When coldToFrozenDir is set, Splunk archives a frozen bucket by keeping only the rawdata directory , because rawdata contains everything needed to rebuild the bucket. (A custom coldToFrozenScript should do the same unless the index stubs out rawdata, which is a metrics-index special case.) Dropping the index files keeps archives small, because they are the larger share of a bucket.
- Step-by-Step Thawing Procedure
- Step 1: Copy the Archived Bucket into thaweddb
Copy the whole archived bucket directory into the index's thawedPath (by default $SPLUNK_DB/<index>/thaweddb ). Make sure its bucket ID does not collide with a bucket already in the index: cp -r /mnt/cold_archive/security/db_1680000000_1670000000_50 \
- $SPLUNK_HOME/var/lib/splunk/security/thaweddb/
- Step 2: Rebuild the Index Files with splunk rebuild
An archived bucket has only rawdata , so it is not searchable yet. splunk rebuild reads the journal and regenerates the tsidx and metadata files: $SPLUNK_HOME/bin/splunk rebuild \
$SPLUNK_HOME/var/lib/splunk/security/thaweddb/db_1680000000_1670000000_50
If the command reports an error, rerun it with the index name as an extra argument ( splunk rebuild <bucket_dir> <index_name> ). Rebuilding decompresses and re-indexes the journal, so it uses significant CPU and disk I/O.
Step 3: Restart the Indexer
Splunk's documented procedure ends with splunk restart , after which the thawed bucket is searchable.
Step 4: Clean Up When Finished
Thawed buckets are outside the aging scheme. They never roll to frozen and do not count toward maxTotalDataSizeMB , so they stay until an administrator deletes them from thaweddb . Restored data also does not count against your license .