ExamFinch.com My Study

Study guide

4.6 Applying a Data Retention Policy

Practice this exam

Splunk Enterprise Certified Admin SPLK-1003

Track your progress as you read

Finished sections get a check mark.

1 Introduction

About the Splunk Enterprise Certified Admin Exam

2 Chapter 1: Splunk Admin Basics

1.1 Splunk Enterprise Architecture & Core Components

3 Chapter 2: License Management

2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting

4 Chapter 3: Splunk Configuration Files

3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations

5 Chapter 4: Splunk Indexes

4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy

6 Chapter 5: Splunk User Management

5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits

7 Chapter 6: Splunk Authentication Management

6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication

8 Chapter 7: Getting Data In

7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder

9 Chapter 8: Distributed Search

8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)

10 Chapter 9: Getting Data In – Staging

9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options

11 Chapter 10: Configuring Forwarders

10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing

12 Chapter 11: Forwarder Management

11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting

13 Chapter 12: Monitor Inputs

12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives

14 Chapter 13: Network and Scripted Inputs

13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection

15 Chapter 14: Agentless Inputs

14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration

16 Chapter 15: Fine Tuning Inputs

15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data

17 Chapter 16: Parsing Phase and Data

16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview

18 Chapter 17: Manipulating Raw Data

17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue

4.6 Applying a Data Retention Policy

Key Takeaways Explain This Topic AI

Data freezes when all events in a bucket are older than frozenTimePeriodInSecs (default 188,697,600 seconds, about 6 years) or when maxTotalDataSizeMB (default 500,000 MB) is exceeded, whichever happens first.

Frozen buckets are deleted by default; setting coldToFrozenDir (or coldToFrozenScript) archives them, keeping only rawdata.

A retention target is only met if the size limits are large enough to hold that much data, so size maxTotalDataSizeMB and volume caps from the expected daily volume.

In an indexer cluster every replicated copy stores rawdata (~15% of raw) but only searchable copies store index files (~35%).

Verify retention with splunk btool indexes list <index> --debug and the Monitoring Console index dashboards.

Last updated: September 2026

Applying a Data Retention Policy

Blueprint objective 4.6 asks you to apply a data retention policy . In Splunk that means deciding how long each index keeps searchable data, how large it may grow, and what happens to buckets when they freeze. It is all configured in indexes.conf on the indexers.

The Retention Toolkit

  • Goal Setting Default
  • Keep data for a period of time frozenTimePeriodInSecs 188697600 (about 6 years)
  • Cap the total size of an index (hot + warm + cold) maxTotalDataSizeMB 500000 (about 500 GB)

Cap the hot/warm or cold directories separately homePath.maxDataSizeMB , coldPath.maxDataSizeMB 0 (not constrained)

Cap a shared storage volume maxVolumeDataSizeMB in [volume:<name>] Not set

Keep frozen data instead of deleting it coldToFrozenDir or coldToFrozenScript Not set (frozen data is deleted )

A frozen bucket is deleted by default . Retention policy is really two decisions: when data freezes, and whether frozen data is archived.

Retention Policy Mechanics: Dual-Trigger Aging

One of the most critical principles tested in Splunk administration is the relationship between size-based aging and time-based aging .

Data aging in Splunk follows a dual-trigger, race-condition model : ┌─────────────────────────────────────────────────┐

  • │ Active Index Storage │
  • │ (Hot + Warm + Cold Buckets) │
  • └───────────────────────┬─────────────────────────┘
  • │
  • ┌──────────────────────────┴──────────────────────────┐
  • ▼ ▼
  • ┌──────────────────────────────────────┐ ┌──────────────────────────────────────┐
  • │ Size-Based Trigger │ │ Time-Based Trigger │
  • │ maxTotalDataSizeMB │ │ frozenTimePeriodInSecs │
  • │ (Total index size exceeds threshold) │ │ (Latest event older than max age) │
  • └──────────────────┬───────────────────┘ └───────────────────┬──────────────────┘
  • │ │
  • └──────────────────────┬──────────────────────┘
  • │
  • WHICHEVER THRESHOLD IS REACHED FIRST
  • │
  • ▼
  • ┌─────────────────────────┐
  • │ Bucket Rolls to Frozen │
  • └─────────────────────────┘
  • The Dual-Trigger Rule

Data rolls to frozen whenever either of the following conditions occurs first:

Size Limit Exceeded: The cumulative size of all hot, warm, and cold buckets exceeds maxTotalDataSizeMB .

Time Limit Exceeded: The latest event timestamp inside a bucket is older than now - frozenTimePeriodInSecs .

Real-World Retention Dilemma: The Premature Freeze

Consider an enterprise compliance policy that legally mandates keeping firewall logs for 365 days (1 year).

  • An administrator configures indexes.conf as follows: [firewall_logs]
  • homePath = /opt/splunk/var/lib/splunk/firewall/db
  • coldPath = /mnt/storage/firewall/colddb
  • thawedPath = /opt/splunk/var/lib/splunk/firewall/thaweddb
  • frozenTimePeriodInSecs = 31536000 # 365 days (1 year)
  • maxTotalDataSizeMB = 500000 # 500 GB

What happens in production?

The organization ingests 50 GB of firewall logs per day.

At roughly 50% of raw volume on disk (rawdata plus index files), 50 GB of daily logs needs ~25 GB of disk space per day.

In only 20 days , total index data reaches 500 GB ( 20 days * 25 GB/day = 500 GB ).

Because maxTotalDataSizeMB has been reached, Splunk immediately freezes the oldest bucket.

Result: Historical data is frozen and removed after only 20 days , violating the 365-day compliance mandate!

Caution

Setting frozenTimePeriodInSecs does not guarantee data will be retained for that duration. If maxTotalDataSizeMB (or a volume quota) is set too low, data will freeze prematurely to protect disk capacity. Administrators must calculate storage capacity accurately to ensure size limits exceed temporal requirements.

Archiving Instead of Deleting

coldToFrozenDir = <path> : Splunk automatically copies frozen buckets to this directory, keeping only the rawdata . It cannot use a volume reference, and changing it requires a restart. If both coldToFrozenDir and coldToFrozenScript are set, coldToFrozenDir takes precedence.

coldToFrozenScript = <script> : Runs a script you supply, which must live in $SPLUNK_HOME/bin or a subdirectory, for custom archiving such as copying to object storage.

Archived data can later be thawed (Chapter 4's bucket lifecycle section): copy the bucket into thaweddb , run splunk rebuild , and restart.

  • Worked Retention Design
  • A security team must keep firewall data searchable for one year and archive it for seven : [firewall]
  • homePath = volume:hot_warm/firewall/db
  • coldPath = volume:cold/firewall/colddb
  • thawedPath = $SPLUNK_DB/firewall/thaweddb
  • frozenTimePeriodInSecs = 31536000 # 365 days searchable
  • maxTotalDataSizeMB = 20000000 # sized so the time limit, not size, drives freezing
  • coldToFrozenDir = /mnt/archive/firewall # keep rawdata for the 7-year archive

Check it after deployment with splunk btool indexes list firewall --debug . The Monitoring Console's index detail dashboards show whether buckets are freezing by age or by size.

Storage Sizing Calculations & Capacity Planning

Designing a Splunk indexing architecture requires calculating disk capacity requirements based on daily ingestion volume, indexing overhead, retention windows, and cluster replication.

Storage Calculation Formula

Splunk's rule of thumb is that compressed rawdata is about 15% of raw volume and the index files are about 35% . In an indexer cluster, every one of the RF copies stores rawdata, but only the SF (searchable) copies also store index files: Cluster disk per day = Raw GB/day × ( 0.15 × R F + 0.35 × S F ) \text{Cluster disk per day} = \text{Raw GB/day} \times (0.15 \times RF + 0.35 \times SF) Cluster disk per day = Raw GB/day × ( 0.15 × RF + 0.35 × SF )

Daily Ingest: Raw, uncompressed volume per day (the same number your license measures).

Rawdata share (~15%): Kept by every replicated copy.

Index-file share (~35%): Kept only by searchable copies. A non-clustered indexer is effectively RF = SF = 1, which gives the familiar "about 50% of raw" rule.

Retention: Days the data must stay searchable (hot + warm + cold).

Concrete Storage Planning Scenario

An enterprise architecture team is designing an indexing cluster with the following requirements:

Daily Ingestion: 1,000 GB/day (1 TB/day)

Retention Requirement: 90 days of searchable active data

Tiering Strategy: 14 days in Hot/Warm (NVMe) + 76 days in Cold (HDD)

Cluster Replication Factor: RF = 3

  • Search Factor: SF = 2 Step 1: Daily cluster-wide disk
  • 1,000 GB x (0.15 x 3 + 0.35 x 2) = 1,000 x (0.45 + 0.70) = 1,150 GB/day
  • Step 2: Hot/Warm tier (14 days)
  • 14 x 1,150 GB = 16,100 GB (~16.1 TB) across the cluster
  • Step 3: Cold tier (76 days)
  • 76 x 1,150 GB = 87,400 GB (~87.4 TB) across the cluster
  • Step 4: Total
  • 16.1 TB + 87.4 TB = ~103.5 TB, before headroom
  • Storage Planning Best Practices

Filesystem Headroom: Always allocate an additional 15–20% of disk space beyond calculated values to accommodate index optimization temporary files, operating system logs, and search dispatch artifacts.

IOPS Allocation: Indexer storage for hot and warm buckets needs fast random I/O. Splunk's reference hardware guidance for indexers calls for storage able to sustain roughly 800 IOPS or more. Cold storage mostly serves historical searches and can be slower.