8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation

Adobe Certified Professional - AEM Sites Developer AD0-E128

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Dispatcher Farm & Multi-Site Configuration

1.1 AEM Dispatcher Configuration & Caching Fundamentals 1.2 Multi-Site Management (MSM) & Blueprint/Live Copy Architecture 1.3 OSGi Configuration Management & Run Modes (.cfg.json)

2 Chapter 2: Oak Indexing & Custom Logging

2.1 Custom Oak Index Definitions & Lucene Query Optimization 2.2 AEM Custom Logging, Logback Configurations & Request Tracing

3 Chapter 3: Component Architecture, HTL & Client Libraries

3.1 Core Components & Proxy Component Pattern 3.2 HTL (HTML Template Language) Block Statements & Context-Aware Escaping 3.3 Client-Side Libraries (Clientlibs) Architecture & Management

4 Chapter 4: Sling Models & OSGi Declarative Services

4.1 Sling Models Framework, Injectors & Model Exporter 4.2 OSGi Declarative Services, Component Lifecycle & Service Rankings

5 Chapter 5: Templates, Fragments & Business Workflows

5.1 Editable Templates, Structure, Initial Content & Content Policies 5.2 Content Fragments, Experience Fragments & Headless Delivery 5.3 AEM Workflows, Custom Process Steps & Launchers

6 Chapter 6: Build, Packaging & Cloud Manager CI/CD

6.1 Maven Project Structure & AEM Project Archetype 6.2 Content Package Types, Packaging Rules & FileVault Validation 6.3 Cloud Manager CI/CD Pipelines & Quality Gates 6.4 Rapid Development Environments (RDE) & Sling Repoinit Scripts

7 Chapter 7: Environment Maintenance & Content Distribution

7.1 AEM Maintenance Releases & Cloud Service Version Updates 7.2 AEM Deployment Topologies & Cloud Architecture 7.3 Replication Agents & Sling Content Distribution 7.4 Repository Maintenance: Revision Cleanup, Compaction & Purge Tasks

8 Chapter 8: Analyzing, Debugging & Diagnostic Tools

8.1 System Log Analysis & Diagnostic Tooling 8.2 Debugging Client-Side Libraries & Granite Dumplibs 8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation 8.4 Environment Troubleshooting, Web Console & Developer Console

8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation

Key Takeaways Explain This Topic AI

The X-Cache-Info HTTP response header and Server-Agent: Communique-Dispatcher disclose internal Dispatcher caching decisions, diagnosing causes such as 'caching disabled', 'stat file is newer', 'no cache rule', or 'authorization header present'.

Unintended cache misses are frequently caused by non-whitelisted URL query parameters (which bypass the cache unless declared in /ignoreUrlParams), authorization headers, session cookies, or matching /rules deny patterns.

Dispatcher cache invalidation scope is governed by statfileslevel, which dictates whether content activations invalidate the entire document root (statfileslevel '0') or isolate invalidation to specific subtree directories.

The AEM as a Cloud Service Dispatcher SDK Docker container provides local validation tooling: bin/validate.sh verifies Apache syntax and structural rules, while bin/docker_run.sh executes the actual Dispatcher image against local AEM author and publish instances.

Requests carrying an HTTP Authorization header or authentication cookies (like login-token) bypass Dispatcher caching by default unless /allowAuthorized '1' is explicitly configured for cacheable public assets.

Last updated: September 2026

8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation

Core Principle: The AEM Dispatcher is the critical caching and security layer standing between end users and the AEM publish tier. A misconfigured Dispatcher leads to catastrophic publish tier overload, stale content delivery, or security vulnerabilities. Diagnosing caching failures requires mastering the X-Cache-Info HTTP response header, analyzing dispatcher.log decision trails, understanding invalidation scopes ( statfileslevel ), and validating Apache configurations locally using the AEM as a Cloud Service Dispatcher SDK .

1. Dispatcher Diagnostic Headers & X-Cache-Info

When troubleshooting caching behavior, developers inspect HTTP response headers emitted by the Dispatcher web server module ( mod_dispatcher.so ). $ curl -I https://wknd.site/us/en/adventures.html

HTTP/1.1 200 OK
Date: Wed, 23 Sep 2026 14:30:15 GMT
Server: Apache/2.4.58 (Unix)
Server-Agent: Communique-Dispatcher
X-Cache-Info: caching disabled; response is not cacheable
Age: 0
Content-Type: text/html;charset=UTF-8
Key Diagnostic Response Headers

Server-Agent: Communique-Dispatcher : Confirms that the HTTP response was routed through an active AEM Dispatcher module rather than hitting the backend AEM Jetty servlet engine directly.

X-Cache-Info : Emitted by Dispatcher when debugging headers are enabled or in development/staging environments. Explains the exact reason why a document was cached, missed, or rejected.

Deciphering X-Cache-Info Status Values

X-Cache-Info Value Technical Meaning & Root Cause

caching disabled The request matched a /cache/rules entry with type "deny" , or the HTTP request method is not GET or HEAD .

stat file is newer A cached file was found in docroot, but its filesystem timestamp ( mtime ) is older than the applicable .stat file , forcing Dispatcher to re-fetch the fresh document from the publish renderer.

no cache rule The request URL did not match any type "allow" rule in /cache/rules . By default, Dispatcher denies caching unless an explicit allow rule matches.

authorization header present The request contained an HTTP Authorization header, triggering automatic cache bypass (unless /allowAuthorized "1" is configured).

response is not 200 OK AEM publish returned a non-200 HTTP response (e.g. 301, 302, 404, 500). Dispatcher will not cache non-200 responses by default.

cache-action: ... Indicates an active caching action (e.g. file written to docroot).

2. Dispatcher Logging & Decision Analysis

To diagnose caching decisions in detail, configure the Dispatcher log level to debug in the Apache virtual host or Dispatcher farm configuration: # In Apache httpd.conf / virtualhost configuration:

LogLevel dispatcher:debug
Anatomy of dispatcher.log Entries

1. Filter Rule Evaluation [Wed Sep 23 14:30:15 2026] [D] [pid 1024] Checking filter rules for [/content/wknd/us/en.html]

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Filter rule [15] matches: /0015 { /type "allow" /url "/content/*" }
[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Filter: ALLOW

2. Cache Lookup & Stat File Invalidation [Wed Sep 23 14:30:15 2026] [D] [pid 1024] Found cached file [/mnt/var/www/html/content/wknd/us/en.html]

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Cache file is older than .stat file [/mnt/var/www/html/content/wknd/.stat]

[Wed Sep 23 14:30:15 2026] [I] [pid 1024] Cache-action: REFETCH from render [publish:4503]

3. Bypassing Cache Due to URL Parameters [Wed Sep 23 14:30:15 2026] [D] [pid 1024] Query string present [utm_source=email]

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] /ignoreUrlParams denied query string -> BYPASS CACHE
3. Diagnosing Chronic Cache Misses

When publish instances suffer high CPU or latency, chronic Dispatcher cache misses are one important cause. Four primary architectural factors cause unintended cache misses:

1. Uncacheable Query Parameters & /ignoreUrlParams

By default, Dispatcher considers any URL containing a query string ( ?key=value ) to be dynamic and bypasses caching entirely. Incoming Request: https://wknd.site/us/en.html?utm_source=newsletter&utm_medium=email

|
v
Dispatcher evaluates /ignoreUrlParams
|

+--> Parameter NOT in /ignoreUrlParams allow list?

| |
| v
| BYPASS CACHE -> Forwards request to AEM Publish
|

+--> Parameter IS in /ignoreUrlParams allow list?

|
v
SERVE FROM CACHE (Returns /mnt/var/www/html/content/wknd/us/en.html)

The Solution: Configuring /ignoreUrlParams In conf.dispatcher.d/cache/default_invalidate.any (or the farm's /cache block), explicitly allow marketing and analytics parameters: /ignoreUrlParams {

/0001 { /glob "*" /type "deny" }
/0002 { /glob "utm_source" /type "allow" }
/0003 { /glob "utm_medium" /type "allow" }
/0004 { /glob "utm_campaign" /type "allow" }
/0005 { /glob "gclid" /type "allow" }
/0006 { /glob "fbclid" /type "allow" }
}
2. Authorization Headers & Session Cookies ( /allowAuthorized )

When a user logs in, the browser sends an Authorization header or a session cookie (e.g. login-token ).

Default Behavior : Dispatcher detects authentication credentials and refuses to cache the response to avoid caching user-specific data into a public shared cache.

The Setting: /allowAuthorized :

/allowAuthorized "0" (Default): Any request with an Authorization header bypasses the cache.

/allowAuthorized "1" : Dispatcher caches responses even if authorization headers are present.

Production Rule : Set /allowAuthorized "1" only for public, non-personalized content trees, or when using Sling Dynamic Include (SDI) to load user-specific fragments via AJAX or SSI/ESI.

3. Invalidation Scope & statfileslevel

When an author publishes a page, AEM's replication agent sends an HTTP POST flush request to Dispatcher. Dispatcher touches the applicable .stat file. Any cached file with a timestamp older than the .stat file timestamp is treated as expired. s t a t f i l e s l e v e l defines the directory depth at which .stat files are created and evaluated. \mathbf{statfileslevel} \text{ defines the directory depth at which .stat files are created and evaluated. } statfileslevel defines the directory depth at which .stat files are created and evaluated. Document Root: /mnt/var/www/html/

├── .stat (statfileslevel 0: Touched on ANY publish)
└── content/
└── wknd/
├── .stat (statfileslevel 2)
├── us/
│ └── en/
│ ├── .stat (statfileslevel 3: Touched on /us/en publish)
│ └── adventure.html
└── fr/
└── fr/
├── .stat (statfileslevel 3: Untouched when /us/en publishes!)
└── adventure.html

statfileslevel Operational Behavior & Invalidation Scope

0 Single .stat file at docroot. Publishing a single page under /content/wknd/us/en invalidates all cached files across all domains and languages in docroot. Causes severe cache churn.

2.stat file at /content/wknd/.stat . Publishing an English page invalidates all language trees under WKND.

3.stat files at /content/wknd/us/.stat . Localizes invalidation to regional language trees. Publishing under /us/en does not invalidate /fr/fr .

Design Guidance: Choose statfileslevel from the actual cache path hierarchy and desired invalidation scope. Test activations across sibling sites and languages; no single level is correct for every structure.

4. HTTP Cache-Control Headers Emitted by AEM

If a custom Sling Servlet, Filter, or Sling Model outputs Cache-Control: private, no-store, no-cache or Pragma: no-cache , Dispatcher honors these directives and refuses to write the response to disk. Use browser DevTools or curl -I to verify that AEM publish returns Cache-Control: max-age=... or standard public headers.

4. Local Validation with AEM as a Cloud Service Dispatcher SDK

In AEM as a Cloud Service, Dispatcher configurations are committed directly to Git and deployed via Cloud Manager pipelines. To prevent deployment pipeline failures, Adobe provides the Dispatcher SDK to validate configurations locally before committing.

SDK Structure & Core Scripts dispatcher-sdk/
├── bin/
│ ├── validate.sh <-- Validates Apache syntax, include directives, and file structure
│ └── docker_run.sh <-- Runs local Apache + Dispatcher in Docker against local AEM
└── src/
└── conf.dispatcher.d/
Phase 1: Static Validation ( bin/validate.sh )
The validate.sh script executes automated linting on the configuration tree: ./bin/validate.sh src/

Validation Checks Performed:

Apache Syntax : Runs httpd -t to ensure all rewrite rules, virtual hosts, and module directives are syntactically valid.

Directive Whitelisting : Verifies that no forbidden Apache directives are declared (e.g. AEMaaCS prohibits ServerName , Listen , and certain unauthorized modules).

Include Structure : Confirms that custom files are included from designated include points ( conf.d/enabled_vhosts/*.vhost , conf.dispatcher.d/enabled_farms/*.farm ).

Phase 2: Local Docker Execution ( bin/docker_run.sh )

Once validation passes, developers execute the local Dispatcher container in Docker, proxying to a local AEM publish instance: # Syntax: bin/docker_run.sh <config-dir> <aem-publish-host:port> <dispatcher-port>

./bin/docker_run.sh src host.docker.internal:4503 8080

Operational Benefits:

Spins up an exact replica of the Cloud Service Apache 2.4 + Dispatcher Linux container.

Directs incoming browser requests at http://localhost:8080 through local Dispatcher caching logic to local AEM publish ( 4503 ).

Generates live container logs directly in the terminal, revealing filter hits, cache creations, and invalidations.

Phase 3: Cloud Manager Pipeline Linting

During Cloud Manager CI/CD pipeline execution, an automated build step runs validate.sh . If any syntax error, broken symlink, or forbidden directive is detected, the pipeline immediately fails at the code quality gate , preventing broken configurations from reaching staging or production.

5. Diagnostic Decision Matrix

Observation / Symptom Root Cause Remediation

X-Cache-Info: stat file is newer Page was invalidated by recent replication flush. Normal behavior after publish; page re-caches on next fetch.

X-Cache-Info: caching disabled URL matches deny rule in /cache/rules or non-GET method. Check /cache/rules in farm file; ensure path has type "allow" .

Cache misses on all marketing traffic ( ?utm_... ). Query parameters not declared in /ignoreUrlParams . Add tracking parameters ( utm_* , gclid ) to /ignoreUrlParams .

Authenticated users experience cache misses on public pages. Request has Cookie or Authorization header. Configure /allowAuthorized "1" under /cache block.

Publishing one page wipes entire site cache. statfileslevel set to 0. Increase statfileslevel to match language folder hierarchy (e.g. 3).

validate.sh reports forbidden Apache directive. Apache configuration contains unauthorized module or rule. Remove prohibited directive to comply with Cloud Service guidelines.