Study guide

11.1 Log Management on Grail: Storage, Ingestion Pipelines & Log Masking

Dynatrace Certified Associate Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Platform Overview, Deployment Models & Architecture

1.1 Dynatrace Platform Capabilities, Web UI Navigation & the Dynatrace Hub 1.2 SaaS vs. Dynatrace Managed Architecture & Security 1.3 Core Telemetry Components: OneAgent, ActiveGate & Grail Lakehouse 1.4 Licensing Models: Dynatrace Platform Subscription (DPS) & Davis Data Units (DDUs)

2 Chapter 2: ActiveGate Architecture & Network Topologies

2.1 ActiveGate Types: Environment vs. Cluster ActiveGates 2.2 ActiveGate Capabilities: Proxying, Private Synthetic, Cloud Integrations & Extensions 2.3 Network Zones, Multi-DC Routing & Firewall Traversal

3 Chapter 3: OneAgent Deployment, Configuration & Management

3.1 OneAgent Architecture, Components & Automatic Injection 3.2 Host Installation on Linux, Windows & Containerized Hosts 3.3 Cloud-Native Deployment: Dynatrace Operator on Kubernetes & OpenShift 3.4 Kubernetes Observability: Clusters, Nodes, Namespaces, Workloads & Events

4 Chapter 4: Monitoring Modes, Process Groups & Infrastructure Observability

4.1 Monitoring Modes: Full-Stack vs. Infrastructure-Only vs. Discovery 4.2 Process Groups, Process Instances & Custom Process Group Detection Rules 4.3 Host Groups, Host Naming Rules & Environmental Metadata Extraction 4.4 Infrastructure & Operations: Host Health, Disk, Network & Availability Analysis 4.5 Database Observability: The Databases App, Connection Pools & Storage Latency

5 Chapter 5: Smartscape Topology & Service Flow

5.1 The 5-Layer Smartscape Topology: Datacenter to Application 5.2 Service Detection: Web Services, Background Tasks & Custom Services 5.3 Service Flow, Backtrace Analysis & Horizontal/Vertical Dependency Mapping

6 Chapter 6: PurePath Distributed Tracing & Code-Level Diagnostics

6.1 PurePath Architecture, Distributed Tracing & Request Context Propagation 6.2 Method Hotspots, CPU Profiling & Memory Allocation Thread Analysis 6.3 Database Service Diagnostics, Slow Queries & Exception Analysis

7 Chapter 7: Davis AI Engine, Baselining & Causation

7.1 Davis Deterministic Causation Engine vs. Pure Correlation 7.2 Automated Dynamic Baselining, Percentiles & Traffic Seasonality 7.3 Event Ingestion, Event Types & Event Hierarchy in Problem Evolution

8 Chapter 8: Problem Lifecycle, Root Cause Analysis & Alerting

8.1 Problem Lifecycle, Severity Categories & Root Cause Cards 8.2 Alerting Profiles, Delay Filters & Management Zone Routing 8.3 Problem Notification Integrations: Webhooks, Slack, ServiceNow & ITSM

9 Chapter 9: Digital Experience Monitoring: Synthetic Monitoring

9.1 Synthetic Monitors: HTTP, Single-Step Browser & Multi-Step Clickpaths 9.2 Private Synthetic Locations, ActiveGate Engines & Credential Vault 9.3 Outage Detection, Performance Baselining & Maintenance Suppression

10 Chapter 10: Digital Experience Monitoring: Real User Monitoring (RUM)

10.1 RUM Architecture, OneAgent Script Injection & Mobile/OpenKit Telemetry 10.2 User Actions: Load vs. XHR/Fetch Actions, Visually Complete & Speed Index 10.3 Apdex Performance Ratings, User Sessions & Session Replay Masking

11 Chapter 11: Log Management and Analytics on Grail with DQL

11.1 Log Management on Grail: Storage, Ingestion Pipelines & Log Masking 11.2 Dynatrace Query Language (DQL): Fetch, Filter, Fields & String Operations 11.3 DQL Aggregations: Summarize, MakeTimeseries, Sorting & Visualizations

12 Chapter 12: Platform Governance, OpenTelemetry & Automation

12.1 Management Zones: Scoping Observability, Dashboards & Access Control 12.2 Tagging Strategies: Automated Rule-Based Tags, Manual Tags & AWS/K8s Metadata 12.3 OpenTelemetry Ingestion, OpenPipeline & Dynatrace Workflows Automation

13 Chapter 13: Cloud Integrations & Agentless Infrastructure Ingestion

13.1 AWS, Azure & Google Cloud Integrations 13.2 VMware vSphere, Extensions & Other Agentless Ingestion Paths

14 Chapter 14: Business Analytics & Business Events

14.1 Business Events: Capture, Ingestion & the bizevents Grail Table 14.2 Business Analytics: Connecting Performance to Revenue, Conversion & KPIs

15 Chapter 15: Notebooks, Dashboards & Data Visualization

15.1 Notebooks: Section-Based Investigation & Shareable Analysis 15.2 Dashboards: Tiles, Variables, Segments & Continuous Monitoring 15.3 Choosing Visualizations: Data Explorer, Chart Types & Metric Selection

16 Chapter 16: Reporting, Metrics & User Behavior Analysis

16.1 Metrics on Grail: Built-in Metrics, Granularity & Retention 16.2 Dashboard Reports, Subscriptions & Exporting Data via APIs 16.3 User Behavior Analysis: Bounce Rate, Landing & Exit Pages, Conversion Goals 16.4 Session Segmentation, User Journeys & Funnel Drop-Off Analysis

17 Chapter 17: Security, Vulnerability Analytics & Application Protection

17.1 Platform Security: Data Privacy, Encryption & Access Control 17.2 Third-Party Vulnerabilities, the Davis Security Score & Security Advisor 17.3 Code-Level Vulnerabilities, Attacks & Runtime Application Protection 17.4 Security Investigator, Notifications & the Remediation Workflow

11.1 Log Management on Grail: Storage, Ingestion Pipelines & Log Masking

Key Takeaways Explain This Topic AI

Dynatrace Grail is a cloud-native, indexless data lakehouse that uses schema-on-read technology and massive parallel processing (MPP) to query petabyte-scale logs without indexing overhead, disk ballooning, or cluster re-indexing.

OneAgent automatically discovers and streams logs from monitored hosts, standard process outputs, systemd-journald, container stdout/stderr, and Windows Event Logs, preserving full topological Smartscape context.

Log storage configurations utilize ingest-time inclusion and exclusion matchers to selectively persist high-value telemetry in Grail while filtering debug noise to optimize Grail storage consumption.

Custom multi-line log detection rules leverage regular expressions to group distributed stack traces, exceptions, and multi-line records into single coherent log events before ingestion.

Ingest-time log masking rules permanently scrub or cryptographically hash PII, credentials, and sensitive cardholder data at the ingestion boundary before disk persistence, ensuring compliance with GDPR, HIPAA, and PCI-DSS.

Last updated: September 2026

Enterprise application architectures generate immense volumes of unstructured log telemetry across microservices, serverless runtimes, container orchestrators, and legacy virtual machines. In traditional log monitoring paradigms, analyzing these massive event streams required deploying complex indexing architectures—most notably Elasticsearch or Lucene-based clusters. These legacy systems depend on inverted indices, pre-defined schema mappings, and continuous shard rebalancing.

However, traditional indexed logging architectures suffer from severe operational and financial bottlenecks:

Index Bloat and Storage Ballooning: Inverted indices frequently consume two to three times more disk storage than the raw log data itself.

Schema Lock-in and Mapping Explosions: Dynamic application logging often introduces new fields that conflict with existing index mappings, causing ingestion failures or dropped records.

Costly Re-Indexing: Modifying retention policies, field data types, or parsing rules historically required re-indexing terabytes or petabytes of historical data, consuming massive compute resources.

Decoupled Telemetry Silos: Logs in legacy systems exist in isolation from infrastructure metrics, distributed traces, and application topology, forcing site reliability engineers (SREs) to manually correlate timestamps during high-severity outages.

Dynatrace eliminates these architectural trade-offs through Dynatrace Grail , an indexless data lakehouse specifically engineered for observability telemetry. Grail unifies logs, metrics, traces, and entity topology into a single causal analytical model powered by Schema-on-Read mechanics and Massive Parallel Processing (MPP) .

1. Grail Lakehouse Architecture: Indexless Storage & Schema-on-Read

At the core of modern Dynatrace Log Management is Grail , a petabyte-scale, column-oriented analytical store built directly on cloud object storage (such as AWS S3, Azure Blob Storage, or Google Cloud Storage). Unlike relational databases or indexed search engines, Grail does not build or maintain inverted indices upon log ingestion. +---------------------------------------------------------------------------------------------------+

  • | GRAIL DATA LAKEHOUSE ARCHITECTURE |
  • +---------------------------------------------------------------------------------------------------+
  • | Ingestion Boundary -> Ingest-Time Rules (Masking, Multi-line, Storage Matchers) |
  • | │ |
  • | ▼ |
  • | Immutable Cloud Object Storage (Indexless, Compressed, Columnar Micro-Partitions) |
  • | │ |
  • | ▼ |
  • | Massively Parallel Processing (MPP) Vectorized Compute Query Nodes |
  • | │ |
  • | ▼ |
  • | Schema-on-Read Evaluation: DQL parses, filters, projects, and aggregates at query runtime |
  • | │ |
  • | ▼ |
  • | Unified Context: Automatic Smartscape Topology Stitching (Host, Process Group, Pod, Service) |
  • +---------------------------------------------------------------------------------------------------+
  • The Schema-on-Read Paradigm

In Grail, logs are ingested and written into compressed, immutable micro-partitions in their raw, native format alongside standard metadata attributes. No index is generated, and no rigid schema is enforced during write operations. Instead, data structure is applied dynamically at read time via the Dynatrace Query Language (DQL) .

This architectural design yields critical operational advantages:

Zero Index Maintenance: Operations teams never manage shards, re-index data, or troubleshoot index mapping explosions.

Infinite Analytical Flexibility: If a developer introduces a new nested JSON attribute or custom logging format, engineers can immediately parse, extract, and aggregate that attribute across historical logs using DQL without retroactive schema migrations.

Radical Cost and Storage Efficiency: By avoiding index generation, Grail achieves superior compression ratios and drastically reduces raw storage overhead compared to legacy search indices.

Native Smartscape & Davis AI Integration

Logs in Grail are never orphaned text strings. Whenever OneAgent or an ActiveGate captures a log event, Dynatrace automatically enriches the log record with contextual topology metadata linking it to the Smartscape dependency model:

dt.entity.host : The unique Dynatrace identifier of the underlying host.

dt.entity.process_group_instance : The specific running process instance emitting the log.

dt.entity.service : The logical software service handling the transaction.

k8s.pod.name , k8s.namespace.name , k8s.container.name : Complete container orchestration context.

Because logs are natively topologically stitched, Davis AI continuously correlates log anomalies (such as sudden surges in ERROR or FATAL log rates) with infrastructure saturation, service response time degradations, and PurePath distributed traces. When Davis raises a Problem card, relevant log patterns are automatically identified as causal or contributory events.

2. Log Ingestion Mechanisms & Discovery

Dynatrace provides multiple ingestion pathways to capture log telemetry across physical data centers, public cloud providers, container clusters, and serverless environments.

OneAgent Automatic Log Detection

When OneAgent is installed on a host in Full-Stack or Infrastructure Monitoring mode, its dedicated log monitoring module automatically discovers, inspects, and streams log data without requiring third-party shippers or manual agent configurations.

OneAgent discovers log sources through several automated detection routines:

Process File Descriptor Tracking: OneAgent monitors active process handles. If a discovered process opens or writes to a file matching standard log file naming conventions (e.g., *.log , catalina.out , error_log ), OneAgent automatically registers the file as a log source.

Standard Directory Inspections: OneAgent routinely monitors standard operating system log directories such as /var/log on Linux and %SystemRoot%\System32\Winevt\Logs on Windows.

Container Output Interception: In containerized environments (Docker, containerd, CRI-O), OneAgent intercepts standard output ( stdout ) and standard error ( stderr ) streams directly from container runtimes or host log mount paths.

Systemd-Journald Integration: On modern Linux systems, OneAgent natively queries the systemd-journald socket to ingest system service units and kernel logs.

Windows Event Log: On Windows hosts, OneAgent integrates directly with the Windows Event Log API to stream Application, System, and Security event channels.

Environment ActiveGate Log Ingestion & Routing

In secure enterprise networks, direct outbound Internet connectivity from application hosts to Dynatrace SaaS is strictly restricted. Environment ActiveGates serve as secure intermediaries:

ActiveGates act as local log proxies, receiving encrypted log batches from OneAgents across internal network zones.

ActiveGates bundle, compress, and stream log telemetry over outbound port 443 to the SaaS Grail lakehouse.

ActiveGates can also act as collection endpoints for syslog daemons (rsyslog, syslog-ng) and cloud log forwarders.

Direct API Ingestion ( POST /api/v2/logs/ingest )

For environments where OneAgent cannot be deployed—such as serverless functions (AWS Lambda, Azure Functions, Google Cloud Run), edge appliances, proprietary network appliances, or third-party log forwarders (Fluentd, Fluent Bit, Logstash, Vector)—Dynatrace provides the high-throughput Log Ingest v2 API ( /api/v2/logs/ingest ).

The Log Ingest API accepts JSON arrays containing plain-text or structured log objects. Ingested payloads can include custom attributes and OpenTelemetry semantic conventions, which Grail automatically ingests into the logs table.

3. Log Ingestion Rules & Storage Configurations

In large enterprise deployments, unconstrained log collection can result in massive volumes of low-value, noisy telemetry (such as verbose health-probe logs or repetitive debug statements) that consume storage capacity and budget. Dynatrace provides Log Storage Rules and Log Processing Rules to give administrators granular control over what telemetry is stored, transformed, or discarded.

Storage Matchers: Inclusion vs. Exclusion

Log storage rules are evaluated at the ingestion boundary before records are committed to persistent Grail storage. Administrators define matchers using boolean expressions based on log attributes: +---------------------------------------------------------------------------------------------------+

  • | LOG STORAGE EVALUATION PIPELINE |
  • +---------------------------------------------------------------------------------------------------+
  • | Incoming Log Stream (OneAgent, ActiveGate, Log Ingest API) |
  • | │ |
  • | ▼ |
  • | Matcher Evaluation: |
  • | - Attribute Matching: log.source, status, content, k8s.namespace.name, dt.entity.host |
  • | ├── MATCH EXCLUSION RULE (e.g., status == "DEBUG" and environment == "staging") |
  • | │ └── ACTION: DROP FROM PERSISTENT STORAGE (No Grail DDU consumed) |
  • | │ |
  • | └── MATCH INCLUSION RULE (e.g., status in ("ERROR", "WARN") or k8s.namespace == "prod") |
  • | └── ACTION: STORE IN GRAIL (Default 35-day or Custom Extended Retention) |
  • +---------------------------------------------------------------------------------------------------+

Storage Inclusion Rules: Explicitly define the criteria for logs that must be persisted in Grail. For example, storing all logs where status in ("ERROR", "WARN", "FATAL") or where k8s.namespace.name == "production" .

Storage Exclusion Rules: Explicitly drop logs matching specific criteria. For example, discarding all records where content like "*healthcheck*" or where status == "DEBUG" originating from non-production host groups.

Stream vs. Storage: Logs excluded from storage do not consume Grail storage licensing. However, administrators can still process them in real time for stream analysis, metric extraction, or event triggering before they are discarded.

Configurable Retention Tiers

By default, logs persisted to Grail are retained for 35 days , which satisfies standard operational baselining, troubleshooting, and post-mortem analysis requirements. However, organizations subject to strict regulatory frameworks (such as financial compliance, HIPAA, or SOC 2) can configure custom retention buckets extending storage for specific log sets up to multiple years.

4. Multi-Line Log Detection & Exception Aggregation

Modern runtime environments frequently output multi-line records. The most prevalent example is an application exception stack trace in Java, .NET, Python, or Go: 2026-09-10 14:32:05.112 ERROR [OrderService] Payment processing failed

  • java.lang.NullPointerException: Cannot invoke "Account.getBalance()" because "userAccount" is null
  • at com.dynatrace.bank.PaymentProcessor.charge(PaymentProcessor.java:142)
  • at com.dynatrace.bank.OrderService.processOrder(OrderService.java:88)
  • at com.dynatrace.bank.OrderController.submit(OrderController.java:45)
  • The Line-Splitting Problem

If a log collector treats every line break ( \n ) as a distinct log record, the single logical exception above is split into five separate log records:

Line 1 is captured as an ERROR log.

Lines 2 through 5 are captured as unassociated, context-free plain text lines, frequently categorized with default INFO or unknown status.

This line-splitting corrupts error baselining, disrupts Davis AI anomaly detection, and makes querying log data in DQL nearly impossible.

Multi-Line Detection Rules

Dynatrace resolves this through Multi-Line Log Detection Rules executed by OneAgent at the point of capture. A multi-line detection rule defines a Header Pattern using regular expressions that delineates the start of a new log record:

Header Pattern Definition: A regex matching the initial timestamp or severity format of a primary log line, such as ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} .

Aggregation Logic: When OneAgent reads a line matching the header pattern, it begins a new log record. Any subsequent lines that do not match the header pattern (such as lines starting with whitespace, tabs, or at com... ) are automatically appended into the content field of the active log record.

Delimiter Options: Administrators can configure OneAgent to detect multi-line records based on standard conventions (e.g., lines starting with dates, times, or custom brackets) or specify explicit start/end pattern boundaries.

5. Log Masking & Sensitive Data Protection

Enterprise logs frequently contain confidential or legally restricted information, including Personally Identifiable Information (PII), authentication credentials, session tokens, and payment data:

  • Primary Account Numbers (credit card PANs)
  • Social Security Numbers (SSNs) and National ID numbers
  • HTTP Authorization: Bearer <token> headers
  • Passwords and secret API keys in query parameters

Failure to sanitize this telemetry violates critical regulatory frameworks including PCI-DSS , GDPR , and HIPAA .

Ingest-Time Masking vs. Query-Time Masking

Understanding the architectural distinction between ingest-time masking and query-time masking is critical for the Dynatrace Certified Associate examination:

Masking Paradigm Execution Stage Storage Impact Security & Compliance Status

Ingest-Time Masking Evaluated at ingestion boundary before writing to Grail Sensitive data is permanently transformed or scrubbed ; raw values never touch persistent disk Fully Compliant (PCI-DSS, GDPR, HIPAA); raw sensitive data cannot be retrieved even via direct storage APIs

Query-Time Masking Evaluated dynamically during DQL query execution in UI Raw sensitive data remains stored in plaintext inside Grail storage; only visually redacted in dashboard views Non-Compliant for strict security mandates; raw data remains vulnerable to unauthorized API queries or data exfiltration

Exam Key Point: Compliance mandates such as PCI-DSS strictly forbid the storage of unencrypted cardholder data on persistent media. Therefore, Ingest-Time Masking is mandatory. Query-time masking does not satisfy persistent storage compliance requirements.

Ingest-Time Masking Mechanics

Administrators configure Log Masking Rules in Dynatrace Settings ( Settings -> Log Monitoring -> Log processing -> Masking ). Each rule defines:

Matcher Scope: An optional filter limiting the rule to specific log sources, hosts, process groups, or Kubernetes namespaces.

Target Expression (Regex): A regular expression matching the sensitive character pattern to be scrubbed.

Masking Action:

String Replacement: Replaces the matched regex group with a static string (e.g., [REDACTED] , [MASKED_SSN] , or XXXX-XXXX-XXXX-1234 ).

Cryptographic SHA-256 Hashing: Replaces the sensitive attribute with its one-way cryptographic SHA-256 hash. Hashing preserves the ability for engineers to correlate distinct user sessions or track transaction flows across microservices without exposing the actual identity or plain-text PII of the user.

Log Ingestion Methods Comparison

  • Attribute OneAgent Auto-Detection Environment ActiveGate Log Ingest v2 API
  • Deployment Location Installed on host / node OS Dedicated proxy VM / host Direct HTTPS endpoint

Discovery Mechanism Automatic via process handles and OS paths Network socket / syslog listener Explicit client HTTP push

Network Path Local agent to ActiveGate/SaaS On-prem DMZ to SaaS Grail Client application to Dynatrace endpoint

Topological Context Full Smartscape (Host, PG, Pod, Service) Network Zone, Host metadata Explicitly declared in payload attributes

Multi-Line Handling Automatic local buffer aggregation Proxy streaming (depends on client) Handled prior to API payload construction

Primary Use Cases Linux/Windows VMs, Kubernetes, OpenShift Network segregation, syslog forwarding Serverless (Lambda), appliances, Fluentd/OTel

Loading diagram... Dynatrace Log Ingestion, Masking, and Grail Lakehouse Pipeline