Study guide
10.3 Normalizing & Validating Data with the CIM App / datamodel command
Splunk Core Certified Power User Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 Exam Facts, Logistics & Test-Taking Strategy 0.2 Splunk Knowledge Objects & Search Pipeline Architecture
2 Chapter 1: Transforming Commands for Visualizations
1.1 The chart Command & Multi-Dimensional Aggregations 1.2 The timechart Command & Time-Series Visualizations 1.3 Transforming Options: limit, useother, usenull, and Split-By Clauses
3 Chapter 2: Filtering and Formatting Results
2.1 The eval Command & Core Evaluation Functions 2.2 Filtering Data: search vs. where Commands 2.3 Handling Missing Data with fillnull
4 Chapter 3: Correlating Events & Transactions
3.1 Understanding Transactions & Event Grouping Concepts 3.2 Configuring transaction Constraints: maxspan, maxpause, startswith, endswith 3.3 Analyzing Transaction Output Fields: duration & eventcount 3.4 transaction vs. stats: Performance & Architectural Trade-offs
5 Chapter 4: Creating and Managing Fields
4.1 Regular Expression Extractions with Field Extractor 4.2 Delimiter-Based Extractions with Field Extractor 4.3 Field Extraction Management, Permissions & Search-Time Precedence
6 Chapter 5: Field Aliases and Calculated Fields
5.1 Creating and Using Field Aliases 5.2 Creating and Configuring Calculated Fields 5.3 Aliases vs. Calculated Fields in the Search Execution Pipeline
7 Chapter 6: Tags and Event Types
6.1 Creating and Using Event Types 6.2 Creating and Managing Tags 6.3 Combining Tags and Event Types for Data Classification
8 Chapter 7: Creating and Using Search Macros
7.1 Creating Basic Search Macros & Backtick Syntax 7.2 Macro Arguments, Validation & Evaluation Functions 7.3 Macro Scoping, Permissions & Troubleshooting
9 Chapter 8: Creating and Using Workflow Actions
8.1 Workflow Action Architecture & Field/Event Context 8.2 Creating GET and POST Workflow Actions 8.3 Creating Search Workflow Actions & Dynamic Secondary Searches
10 Chapter 9: Data Models and Pivot
9.1 Data Model Architecture & The Pivot Tool 9.2 Creating and Structuring Datasets & Hierarchies 9.3 Accelerating Data Models & TSIDX Acceleration
11 Chapter 10: Common Information Model (CIM)
10.1 The Splunk Common Information Model (CIM) & Normalization 10.2 CIM Knowledge Objects, Data Models & Schemas 10.3 Normalizing & Validating Data with the CIM App / datamodel command
10.3 Normalizing & Validating Data with the CIM App / datamodel command
Key Takeaways Explain This Topic AI
CIM compliance can be validated in SPL using tag searches ( tag=<cim_tag> ), the generating command | datamodel <model> <dataset> search , and the clean syntax | from datamodel:<model>.<dataset> .
Accelerated CIM data models achieve maximum performance when queried using | tstats summariesonly=t ... from datamodel=<model>.<dataset>, reading directly from pre-built TSIDX summary files.
The CIM Setup interface (in Splunk_SA_CIM) enables administrators to restrict Data Model Acceleration to specific indexes, preventing acceleration jobs from scanning irrelevant indexes and conserving indexer resources.
Normalizing a new proprietary data source to CIM follows a 6-step methodology: (1) Ingest & extract fields, (2) Create field aliases, (3) Create calculated fields, (4) Define event types, (5) Apply CIM tags, and (6) Validate via Pivot/tstats.
Common CIM normalization failures include missing tags in tags.conf, uppercase field names breaking lowercase requirements, unmapped action values, and un-rebuilt acceleration summaries after configuration changes.
Last updated: August 2026
10.3 Normalizing & Validating Data with the CIM App / datamodel command
Quick Summary: Configuring field extractions, field aliases, and tags is only the initial phase of data integration. Verifying and validating that data models accurately ingest, structure, and accelerate normalized events is critical for production reliability. Incomplete mappings, casing discrepancies, or missing tags cause security correlation searches and monitoring dashboards to silently fail. This section details how to query CIM data models using native SPL commands, configure index acceleration constraints in the CIM Add-on Setup interface, execute a complete step-by-step normalization workflow on a proprietary data source, and troubleshoot common CIM failure modes.
1. Methods for Querying and Validating CIM Data Models in SPL
Splunk provides four primary mechanisms to query and validate CIM data models in Search Processing Language (SPL): +-----------------------------------------------------------------------------------------+
- | SPL CIM QUERY & VALIDATION METHODS |
- | |
- | 1. Tag-Based Search: tag=<cim_tag> |
- | -> Queries raw events matching CIM tags; validates tag assignment & event types |
- | |
- | 2. | datamodel <Model> <Dataset> search |
- | -> Standard generating command; returns events with Dataset.field prefixes |
- | |
- | 3. | from datamodel:"<Model>"."<Dataset>" |
- | -> Clean generating syntax; automatically strips dataset prefixes from field names |
- | |
- | 4. | tstats summariesonly=t <func> from datamodel=<Model>.<Dataset> by <field> |
- | -> High-speed accelerated query; reads pre-built TSIDX summaries from disk |
- +-----------------------------------------------------------------------------------------+
- 1. Tag-Based Search ( tag=<cim_tag> )
The most direct way to confirm whether events from a specific sourcetype are tagged for CIM ingestion is an ad-hoc tag search: `-- Audit tag assignments on Cisco ASA firewall logs --`
- index=network sourcetype="cisco:asa"
- | stats count by eventtype, tag
If the returned tag values do not include network and communicate , the events are invisible to the Network_Traffic data model.
2. The | datamodel Command
The | datamodel command is a generating command used to inspect data model schemas or extract dataset events.
Schema Inspection (No search mode specified): | datamodel Authentication
Returns a JSON-formatted structural definition of the Authentication data model, including all dataset names, constraints, and mapped fields.
- Event Data Query ( search mode): | datamodel Authentication Authentication search
- | search Authentication.action="failure"
- | stats count by Authentication.user, Authentication.src
Important
Prefixed Field Names in | datamodel : When using | datamodel <model> <dataset> search , all returned fields are prefixed with the dataset hierarchy name (e.g., Authentication.user , Authentication.src , Authentication.action ). Any downstream SPL filtering or statistical commands must reference the fully qualified prefixed name unless explicitly renamed using | rename Authentication.* AS * .
3. The Modern | from datamodel: Syntax
Splunk supports the | from datamodel: syntax, providing a streamlined, SQL-like interface that automatically removes dataset prefixes and returns clean field names: `-- Query failed authentications directly without field prefixes --`
- | from datamodel:"Authentication"."Failed_Authentication"
- | stats count AS failed_logins by user, src, app
- | sort - failed_logins
Targeting a child dataset (such as Failed_Authentication ) automatically applies that child's constraints ( action=failure ), eliminating the need to write manual filter clauses.
4. High-Performance Accelerated Queries with | tstats
When Data Model Acceleration (DMA) is enabled, Splunk indexers compile data model fields into optimized .tsidx summary files. The | tstats (transforming stats) command queries these .tsidx files directly without reading raw journal files from disk, executing hundreds of times faster than standard search pipelines. `-- High-speed network traffic analysis using accelerated summaries --`
- | tstats summariesonly=t count, sum(All_Traffic.bytes) AS total_bytes
- from datamodel=Network_Traffic.All_Traffic
- where All_Traffic.action="allowed"
- by All_Traffic.src_ip, All_Traffic.dest_ip, All_Traffic.dest_port
- | rename All_Traffic.* AS *
- | eval total_mb = round(total_bytes / (1024 * 1024), 2)
- | sort 20 - total_mb
- Key tstats Execution Parameters
summariesonly=t : Restricts the search exclusively to accelerated .tsidx summary data. If data is unaccelerated or falls outside the acceleration window, it is omitted. This guarantees sub-second execution speeds for Enterprise Security correlation searches.
summariesonly=f (Default): Queries accelerated summaries where available and falls back to scanning raw index data for unaccelerated time periods.
2. The CIM Setup Interface & Acceleration Governance
In enterprise environments processing terabytes of data daily, unconstrained data model acceleration can exhaust indexer storage and CPU capacity. The CIM Add-on Setup Interface provides centralized governance over acceleration scope. +-----------------------------------------------------------------------------------------+
- | CIM ADD-ON SETUP INTERFACE ARCHITECTURE |
- | |
- | Navigation: Manage Apps > Splunk_SA_CIM > Set up (or CIM Setup App) |
- | |
- | [ DATA MODEL ACCELERATION CONFIGURATION ] |
- | Data Model Acceleration Status Summary Range Index Whitelist |
- | Authentication [ Enabled [V] ] [ 90 Days [V] ] [ win_auth, okta, vpn ] |
- | Network_Traffic [ Enabled [V] ] [ 30 Days [V] ] [ firewall, netflow ] |
- | Web [ Enabled [V] ] [ 30 Days [V] ] [ proxy, web_servers ] |
- | Endpoint [ Disabled [V] ] [ 7 Days [V] ] [ edr_logs ] |
- +-----------------------------------------------------------------------------------------+
- Index Whitelisting (Constraining Acceleration Scope)
By default, a data model search constraint like tag=authentication scans all indexes configured on the Splunk deployment. If non-authentication data resides in an unconstrained index, indexers waste CPU cycles evaluating events.
Through the CIM Setup interface (or datamodels.conf and macros.conf ), administrators define Index Whitelists for each data model:
For Authentication , restrict search scope to index=wineventlog OR index=okta OR index=vpn .
For Network_Traffic , restrict search scope to index=firewall OR index=paloalto OR index=netflow .
Monitoring Acceleration Status via REST API
Administrators can inspect summarization completeness and disk usage using SPL: | rest /services/admin/summarization by_subpath=t
- | search id="*Network_Traffic*" OR id="*Authentication*"
- | table summary.id, summary.complete, summary.size, summary.earliest_time, summary.latest_time
- 3. Step-by-Step Methodology: Normalizing a Proprietary Data Source
To demonstrate the end-to-end normalization methodology, let us walk through normalizing a custom corporate web proxy ( sourcetype = custom:corp:proxy ) to the CIM Web data model.
Sample Raw Event 2026-08-24 15:45:12.102 CLIENT=10.2.14.55 SERVER=198.51.100.80 VERB=GET URI="/login/auth.php?user=jsmith" CODE=200 BYTES_OUT=450 BYTES_IN=2850 ELAPSED_MS=45 AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
- Step 1: Ingest & Extract Fields ( props.conf )
- Ensure all vendor-native tokens are extracted into field names: [custom:corp:proxy]
- # Key-value pairs extract automatically; regex used for structured headers if needed
- KV_MODE = auto
- Step 2: Create Field Aliases to CIM Standard Names ( props.conf )
Map vendor-specific keys to standard CIM field names ( src , dest , http_method , status , http_user_agent ): [custom:corp:proxy]
- FIELDALIAS-cim_endpoints = CLIENT AS src_ip CLIENT AS src SERVER AS dest_ip SERVER AS dest
- FIELDALIAS-cim_http = VERB AS http_method CODE AS status AGENT AS http_user_agent
- FIELDALIAS-cim_bytes = BYTES_OUT AS bytes_out BYTES_IN AS bytes_in
- Step 3: Create Calculated Fields for Derived Attributes ( props.conf )
- Compute composite metrics, normalize action verbs, and calculate durations: [custom:corp:proxy]
- EVAL-bytes = bytes_in + bytes_out
- EVAL-duration = ELAPSED_MS / 1000
- EVAL-action = if(status < 400, "allowed", "blocked")
- EVAL-url = "http://" + dest + URI
- EVAL-uri_path = replace(URI, "\?.*$", "")
- EVAL-uri_query = if(match(URI, "\?"), replace(URI, "^[^\?]*\?", ""), null())
- EVAL-vendor_product = "Custom Corporate Proxy"
- Step 4: Define Event Types for Specific Activities ( eventtypes.conf )
- Group the proxy events into a classification event type: [custom_corp_proxy_web]
- search = sourcetype="custom:corp:proxy" status=*
- Step 5: Apply CIM Tags to Event Types ( tags.conf )
Attach the mandatory web and proxy tags so the events satisfy the Web data model root dataset constraints: [eventtype=custom_corp_proxy_web]
- web = enabled
- proxy = enabled
- Step 6: Validate in Pivot & tstats
Validate that the events are accessible in Pivot and return accurate metrics via accelerated tstats : | tstats summariesonly=f count, sum(Web.bytes) AS total_bytes
- from datamodel=Web.Web
- where Web.sourcetype="custom:corp:proxy"
- by Web.src, Web.dest, Web.http_method, Web.action, Web.status
4. Troubleshooting Common CIM Normalization Pitfalls +-----------------------------------------------------------------------------------------+
- | CIM NORMALIZATION TROUBLESHOOTING MATRIX |
- | |
- | Symptom: Events exist in raw index, but | datamodel returns 0 results. |
- | Root Cause: Missing or disabled tags in tags.conf for the event type. |
- | Remediation: Verify eventtypes.conf matches events; enable required tags in tags.conf. |
- | |
- | Symptom: | tstats returns null or blank columns for an extracted field. |
- | Root Cause: Field name contains uppercase letters (e.g., Src_IP AS Src_IP). |
- | Remediation: Ensure FIELDALIAS target names are strictly lowercase (pan_src AS src_ip).|
- | |
- | Symptom: Events missing from child dataset (e.g., Web.Proxy or Traffic_By_Action). |
- | Root Cause: Action field contains unmapped vendor strings (e.g., "Pass" vs "allowed").|
- | Remediation: Add EVAL-action with case() or if() to normalize to standard values. |
- | |
- | Symptom: Historical searches via | tstats do not reflect new field aliases. |
- | Root Cause: Accelerated .tsidx summaries contain old schema prior to alias update. |
- | Remediation: Navigate to Settings > Data Models > Edit Acceleration > Rebuild. |
- +-----------------------------------------------------------------------------------------+
- Detailed Troubleshooting Guidance
Auditing Missing Tags: Always inspect raw tag assignments before debugging field extractions. A data model's root constraint is almost universally tag-based. If tag=web is missing, Splunk drops the events before evaluating any field aliases.
Case Sensitivity Discrepancies: Splunk field names are case-sensitive. If an alias defines AS Dest_Port instead of AS dest_port , the CIM schema will treat dest_port as null.
Data Model Acceleration Rebuilds: Modifying props.conf or tags.conf does not automatically update existing historical .tsidx summary files on indexers. After updating CIM knowledge objects in production, administrators must trigger a Data Model Rebuild in the Data Models management interface.