Study guide
14.2 HTTP Event Collector (HEC) Architecture & Administration
Splunk Enterprise Certified Admin SPLK-1003
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Splunk Enterprise Certified Admin Exam
2 Chapter 1: Splunk Admin Basics
1.1 Splunk Enterprise Architecture & Core Components
3 Chapter 2: License Management
2.1 License Types, Stacks, Pools & Licensing Architecture 2.2 License Violations, Rolling Window Warnings & Troubleshooting
4 Chapter 3: Splunk Configuration Files
3.1 Splunk Configuration Directory Structure 3.2 Configuration Layering, Precedence & Resolution Rules 3.3 Using btool to Inspect & Debug Configurations
5 Chapter 4: Splunk Indexes
4.1 Index Structure: Directories, Buckets & Bucket Files 4.2 Bucket Types & the Hot-Warm-Cold-Frozen-Thawed Lifecycle 4.3 Checking Index Data Integrity 4.4 indexes.conf Options: Paths, Bucket Sizing & Volumes 4.5 The Fishbucket: File Tracking, CRC Collisions & Re-reading Files 4.6 Applying a Data Retention Policy
6 Chapter 5: Splunk User Management
5.1 Splunk Roles, Capabilities & Custom Role Design 5.2 User Account Administration, Search Restrictions & Limits
7 Chapter 6: Splunk Authentication Management
6.1 Integrating Splunk with LDAP 6.2 Other Authentication Options & Enabling Multifactor Authentication
8 Chapter 7: Getting Data In
7.1 CLI Administration & Basic Data Onboarding Workflows 7.2 Forwarder Types: Universal Forwarder vs. Heavy Forwarder
9 Chapter 8: Distributed Search
8.1 How Distributed Search Works 8.2 Roles of the Search Head and Search Peers 8.3 Configuring Search Peers & Distributed Search Groups 8.4 Search Head Scaling & Search Head Clustering (SHC)
10 Chapter 9: Getting Data In – Staging
9.1 Ingestion Pipeline Stages: Input, Parsing, Merging, Typing & Indexing 9.2 Pipeline Component Placement & Queue Management 9.3 Splunk Input Options
11 Chapter 10: Configuring Forwarders
10.1 Forwarder Setup & outputs.conf Load Balancing 10.2 Indexer Acknowledgment, Data Cloning & Forwarder Routing
12 Chapter 11: Forwarder Management
11.1 Deployment Management & the Deployment Server 11.2 Deployment Apps & Deployment Clients 11.3 Server Classes, Client Grouping & App Deployment Strategies 11.4 Forwarder Management Monitoring & Troubleshooting
13 Chapter 12: Monitor Inputs
12.1 File & Directory Monitoring Mechanics 12.2 Monitor Input Tuning: Whitelists, Blacklists & CRC Settings 12.3 Deploying a Remote Monitor Input, Network Shares & Archives
14 Chapter 13: Network and Scripted Inputs
13.1 Creating Network Inputs: TCP & UDP 13.2 Optional Settings for Network Inputs 13.3 Scripted Inputs & PowerShell Data Collection
15 Chapter 14: Agentless Inputs
14.1 Windows Management Instrumentation (WMI) Inputs 14.2 HTTP Event Collector (HEC) Architecture & Administration
16 Chapter 15: Fine Tuning Inputs
15.1 Input-Phase Processing & Default Metadata Assignment 15.2 Character Set Encoding & Handling Binary Data
17 Chapter 16: Parsing Phase and Data
16.1 Line Breaking & Event Segmentation 16.2 Timestamp Extraction, Time Zones & Clock Skew Handling 16.3 Validating Event Parsing with Data Preview
18 Chapter 17: Manipulating Raw Data
17.1 Transformation Mechanics: props.conf & transforms.conf 17.2 Data Masking, Redaction & SEDCMD 17.3 Event Routing, Dynamic Index Assignment & Filtering to nullQueue
14.2 HTTP Event Collector (HEC) Architecture & Administration
Key Takeaways Explain This Topic AI
HEC listens on port 8088 by default and is disabled globally (disabled = 1) until enabled; tokens live in [http://<token_name>] stanzas, usually in the splunk_httpinput app.
A token's indexes setting lists the indexes its events may use; if it is not set, any index can be used, and an event naming a disallowed index gets code 7, Incorrect index.
The event endpoint takes JSON with an event key plus optional time, host, source, sourcetype, index, and fields; the raw endpoint takes plain text with metadata from the query string or token.
Clients authenticate with Authorization: Splunk <token>, basic authentication, or ?token= when allowQueryStringAuth = true; codes include 4 Invalid token, 9 Server is busy, and 10 Data channel is missing.
HEC indexer acknowledgment (useACK = true on the token) requires a channel GUID; a true ack status means the data was replicated at the desired replication factor, and it differs from forwarder useACK.
Last updated: September 2026
HTTP Event Collector (HEC)
Quick Summary: The HTTP Event Collector (HEC) lets applications send events to Splunk over HTTP or HTTPS with a token instead of a user name and password, so no forwarder is needed on the client. HEC listens on port 8088 by default, is disabled globally until you enable it, and is configured with a global [http] stanza plus one [http://<token_name>] stanza per token. Know the JSON event endpoint versus the raw endpoint, how tokens restrict indexes, the status codes clients receive, and how HEC indexer acknowledgment uses channels.
What HEC Is and Where It Runs
HEC is an input built into splunkd . Any client that can make an HTTP POST can send data, for example applications, container logging drivers, serverless functions, and automation scripts.
Token authentication: each token is a GUID. Clients present it in the Authorization header, and tokens can be enabled, disabled, or restricted to specific indexes without sharing user credentials.
Where to run it: HEC can run on a standalone instance, on indexers, or on heavy forwarders. At scale it is common to place several HEC instances behind a load balancer.
Where tokens are stored: tokens created in Splunk Web are written to $SPLUNK_HOME/etc/apps/splunk_httpinput/local/inputs.conf . To manage tokens centrally, useDeploymentServer = 1 writes the HEC configuration to the deployment server repository instead.
Enabling it: in Splunk Web, open Settings > Data inputs > HTTP Event Collector > Global Settings and enable all tokens. This sets disabled = 0 in the [http] stanza.
Global Settings: the [http] Stanza
- Setting Default Meaning
- disabled 1 HEC is off until you set 0
- port 8088 Listening port
enableSSL true Use TLS. HEC shares TLS settings with the management port and cannot use TLS if the management server has TLS turned off
- serverCert $SPLUNK_HOME/etc/auth/server.pem Server certificate
- dedicatedIoThreads 0 Dedicated I/O threads; 0 means the input uses a single thread
- maxSockets 0 Simultaneous connections; 0 = one third of the host's open-file limit (minimum 50)
- maxThreads 0 Threads for active HTTP transactions; 0 = one third of the host's thread limit (minimum 20)
- maxEventSize 5MB Larger events trigger a parsing error
- ackIdleCleanup / maxIdleTime true / 600 Remove ACK channels idle for 10 minutes
channel_cookie empty Return the channel ID in a cookie, for load balancers that can only make sessions sticky on cookies
A separate limit, max_content_length in the [http_input] stanza of limits.conf , caps the size of a request body (default 838,860,800 bytes, about 800 MB). [http]
- disabled = 0
- port = 8088
- enableSSL = 1
- dedicatedIoThreads = 2
- Token Stanzas: [http://<token_name>]
- Setting Default Meaning
- token none The GUID that clients send
- disabled 0 Disable one token without affecting the others
- index the default index Where events go when they do not name an index
- indexes empty = any index Comma-separated list of indexes that events for this token may use
- sourcetype empty Default source type when the event does not set one
- connection_host not set ip , dns , proxied_ip (use the X-Forwarded-For header), or none
- useACK false Enable HEC indexer acknowledgment for this token
- allowQueryStringAuth false Allow the token in the URL ( ?token= )
- queueSize / persistentQueueSize 500KB / 0 Input queue sizes
- Restricting Indexes with indexes
An event can name its own index in the JSON metadata. That index must be in the token's indexes list if the list is set; otherwise the request fails with status code 7, Incorrect index . The key trap: if indexes is not set, the list is empty and any index can be used. Set indexes on every token that untrusted or shared clients use. [http://k8s_production_ingress]
- disabled = 0
- token = 7a8b9c0d-1e2f-3a4b-5c6d-7e8f9a0b1c2d
- index = prod_k8s
- indexes = prod_k8s, prod_k8s_security
- sourcetype = kube:ingress
- useACK = 1
- [http://cicd_build_runner]
- disabled = 0
- token = 99887766-5544-3322-1100-aabbccddeeff
- index = cicd_events
- indexes = cicd_events
- sourcetype = gitlab:pipeline
- Authentication Options
- Method How the client sends the token
- HTTP header (standard) Authorization: Splunk <token>
- Basic authentication Any user name with the token as the password, e.g. curl -u "x:<token>"
Query string ?token=<token> . Works only when the token has allowQueryStringAuth = true ; tokens in URLs can be logged in clear text, so always use HTTPS
If a token is sent in both the query string and the header, the query-string token takes precedence.
- Event Endpoint vs. Raw Endpoint
- /services/collector (also /services/collector/event ): JSON Events {
- "time": 1727082400.451,
- "host": "k8s-worker-node-04",
- "source": "auth-svc",
- "sourcetype": "auth:json",
- "index": "prod_k8s",
- "event": {"action": "login_attempt", "user": "jsmith", "status": "success"},
- "fields": {"environment": "production", "region": "us-east-1"}
- }
event holds the data: a string, a number, or a JSON object. The metadata keys time , host , source , sourcetype , index , and fields are optional; anything left out takes the token's values.
time is epoch time in <sec>.<ms> format.
fields defines a flat set of indexed fields . It is accepted only by the event endpoint and does not apply to raw data.
Batching: put several event objects one after another, or in a JSON array, in one request to improve throughput.
- /services/collector/raw : Raw Text curl -H "Authorization: Splunk 7a8b9c0d-1e2f-3a4b-5c6d-7e8f9a0b1c2d" \
- H "X-Splunk-Request-Channel: 18654C68-B28B-4450-9CF0-6E7645CA60CA" \
- "https://hec.corp.example:8088/services/collector/raw?index=net_fw&sourcetype=cisco:asa&host=edge-fw01" \
- d "Mar 23 03:00:00 edge-fw01 %ASA-6-302013: Built inbound TCP connection 10423"
The body is plain text. HEC breaks it into events and sends them through the pipeline, which extracts timestamps and applies the source type's props.conf rules.
Metadata comes from the query string , the token, or the global settings, and applies to all events in the request.
Splunk's examples for the raw endpoint include a channel identifier ( X-Splunk-Request-Channel header or ?channel= ), and a channel is mandatory whenever the token uses indexer acknowledgment.
- Feature Event endpoint Raw endpoint
- Body JSON objects with an event key Plain text
- Per-event metadata In the JSON Query string, token, or global defaults
- Indexed fields ( fields ) Supported Not applicable
- Event breaking Each JSON object is an event Line-breaking rules for the source type
- Status Codes Returned to Clients
HEC replies with an HTTP status and a JSON body containing a Splunk status code. The codes most worth knowing:
- Code HTTP Message
- 0 200 Success
- 1 403 Token disabled
- 2 401 Token is required
- 3 401 Invalid authorization
- 4 403 Invalid token
- 5 400 No data
- 6 400 Invalid data format
- 7 400 Incorrect index
- 8 500 Internal server error
- 9 503 Server is busy
- 10 400 Data channel is missing
- 11 400 Invalid data channel
- 12 400 Event field is required
- 13 400 Event field cannot be blank
- 14 400 ACK is disabled
Newer codes cover the health endpoint (17–20, for example 18 "HEC is unhealthy, queues are full") and capacity warnings (24–27, including HTTP 429 when the queue or ACK channel is at capacity). Splunk cautions that clients must act on these codes, for example by retrying after 503 or 429, or data can be lost.
HEC Indexer Acknowledgment and Channels
By default, HEC returns HTTP 200 as soon as the request appears valid, before the data enters the processing pipeline . An outage after that point can lose the data. HEC indexer acknowledgment lets the client check what happened. It is a different feature from forwarder indexer acknowledgment ( useACK in outputs.conf ), even though the names are the same.
Enable it per token with useACK = true (or the Enable indexer acknowledgment checkbox when creating the token).
Send a channel with every request: a GUID in X-Splunk-Request-Channel or ?channel= . Without it, HEC returns code 10, Data channel is missing . One channel per client stops a fast client from slowing a slow one down.
Receive an ack ID: each request returns an identifier such as {"ackId": 42} .
Query the status by posting {"acks": [40, 41, 42]} to /services/collector/ack on the same channel.
Interpret the reply: true means the events for that ID were replicated at the desired replication factor . It does not guarantee indexing, because the parsing pipeline can still drop events it cannot parse. false means there is no status yet or no information for that ID.
Stop querying after true : HEC deletes the status once it has returned true , so asking again returns false . Resend data whose ID never becomes true within a reasonable time.
Load Balancers
Acknowledgment status is kept by the HEC instance that received the request, so ACK queries must reach the same instance. Configure the load balancer for sticky sessions on the channel. For load balancers that can only stick on cookies (Splunk's example is AWS ELB), set channel_cookie so that HEC returns the channel in a cookie.