Study guide
17.2 Log Routing, Export Destinations, and Exclusions
Google Cloud Professional Cloud Developer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam and Compute Platform Decisions
1.1 Current Exam, Sources, and Scenario Method 1.2 Cloud Run Execution and Platform Selection 1.3 GKE Modes and Compute Engine Trade-offs
2 Chapter 2: Container Engineering and API Contracts
2.1 Stateless Microservices and Multi-stage Builds 2.2 Container Hardening, Signals, and Layer Caching 2.3 REST, gRPC, and Protobuf Contracts
3 Chapter 3: gRPC Compatibility and Geographic Routing
3.1 Schema Evolution and gRPC Routing 3.2 Geography, Network Tiers, and Load Balancers 3.3 Serverless NEGs, Sessions, and Edge Controls
4 Chapter 4: API Management and Caching
4.1 API Management & Rate Limiting with Apigee and API Gateway 4.2 Memorystore and Cache Access Patterns 4.3 Cache Invalidation, Capacity, and Failover
5 Chapter 5: Messaging, Task Dispatch, and Orchestration
5.1 Event-Driven Messaging with Cloud Pub/Sub 5.2 Task Queues & Rate Throttling with Cloud Tasks 5.3 Event Routing and Service Orchestration: Eventarc & Cloud Workflows
6 Chapter 6: Resilience and Relational Availability
6.1 Designing Resilient Systems: Circuit Breaking, Retries & Failover 6.2 Cloud SQL Availability and Recovery 6.3 Connection Budgets and Spanner Consistency
7 Chapter 7: Distributed Schemas and NoSQL Data Models
7.1 Spanner Keys, Indexes, and Transactions 7.2 Firestore Data, Indexes, and Transactions 7.3 Bigtable Row Keys and Replication
8 Chapter 8: Object Storage and Application Identity
8.1 Cloud Storage Lifecycle and Signed Access 8.2 Workload Identity and Service Authentication 8.3 Secret Delivery, Rotation, and IAP
9 Chapter 9: Security, Data Integration, and Developer Credentials
9.1 Private Communication, Key Rotation, and Vulnerability Response 9.2 AlloyDB Schemas and BigQuery Ingestion 9.3 CLI Configurations, Local ADC, and Cloud Shell
10 Chapter 10: Managed Development and Automated Tests
10.1 Cloud Workstations and the Container Inner Loop 10.2 Emulating Google Cloud Services & Database Auth Proxies 10.3 Automated Testing Strategies: Unit, Integration, and Mocking
11 Chapter 11: AI Development Tools and Build Pipelines
11.1 AI Coding Assistants, Context, MCP, and Test Review 11.2 Cloud Build Workers and Step Dependencies 11.3 Builders, Substitutions, Triggers, and IAM
12 Chapter 12: Artifact Security and Admission Policies
12.1 Artifact Management and Vulnerability Scanning with Artifact Registry 12.2 Build Provenance and Admission Concepts 12.3 Attestors, Policy Enforcement, and Break-glass
13 Chapter 13: Cloud Run and GKE Deployment Configuration
13.1 Cloud Run Deployment and Scaling Configuration 13.2 Cloud Run CPU, Private Egress, and Secrets 13.3 GKE Workload Controllers and Gateway Routing
14 Chapter 14: Kubernetes Resources, Health, and Progressive Releases
14.1 GKE Resources, OOM Diagnosis, and Autoscaling 14.2 Container Health Probes, Lifecycle Hooks & Graceful Shutdown 14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
15 Chapter 15: Source/Event Deployment and Cloud API Integration
15.1 Source Deployment, Event Receivers, and API Versions 15.2 Client Libraries, Service Enablement, and ADC 15.3 API Responses, Pagination, Caches, and Retries
16 Chapter 16: Tracing, Sampling, and Monitoring
16.1 Trace Context and OpenTelemetry Export 16.2 Custom Spans, Trace Analysis, and Sampling 16.3 Application Metrics, Health Checks & Alerting with Cloud Monitoring
17 Chapter 17: Structured Logs, Routing, and Error Management
17.1 Structured Logs, Trace Correlation, and Queries 17.2 Log Routing, Export Destinations, and Exclusions 17.3 Error Reporting Formats and Regression Handling
18 Chapter 18: Profiling, Generative AI, and Assisted Troubleshooting
18.1 Cloud Profiler Types and Flame Graph Analysis 18.2 Generative AI Integration with the Google Gen AI SDK 18.3 Evidence-Based Troubleshooting with AI Assistance
17.2 Log Routing, Export Destinations, and Exclusions
Key Takeaways Explain This Topic AI
Cloud Storage supports archival, BigQuery exported-log SQL and Pub/Sub downstream streaming.
Cloud Logging Log Analytics is a log-bucket feature distinct from a BigQuery export sink.
Apply narrow exclusions to benign telemetry while retaining errors and required audit records.
Last updated: October 2026
Log Router & Log Sinks Architecture
Every log entry emitted by an application or Google Cloud service enters the Log Router . The Log Router evaluates the entry against defined sink filters and exclusion rules before deciding where the log should be delivered. +-------------------------------------+
- | Application Log Entry |
- | (stdout / JSON) |
- +-------------------------------------+
- |
- v
- +-------------------------------------+
- | Log Router |
- | - Evaluates inclusion sink filters |
- | - Evaluates bucket exclusion rules |
- +-------------------------------------+
- / | | \
- / | | \
- +----------+ +-----------+ +----------+ +-------------------+
- | Cloud | | BigQuery | | Pub/Sub | | Cloud Logging |
- | Storage | | (Log | | (SIEM / | | Buckets |
- | (Archive)| | Analytics)| | Splunk) | | (_Default / _Req) |
- +----------+ +-----------+ +----------+ +-------------------+
- Sink Destinations and Use Cases
Log Router sinks route entries to supported destinations, with destination-specific batching and delivery latency:
Cloud Storage ( storage.googleapis.com/[BUCKET_NAME] ):
Primary Use Case: Long-term archival and compliance retention (e.g., preserving audit logs for an explicitly applicable retention requirement).
Operational Profile: Batched JSON export; delivery is not immediate. Select the storage class from retention, access frequency and retrieval costs.
BigQuery ( bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET_ID] ):
Primary Use Case: SQL export analysis and ad-hoc querying, business intelligence dashboards, and cross-referencing system logs with business database tables.
Operational Profile: Streaming export into the configured table organization; date-sharded or partitioned behavior and clustering are not automatic universal defaults.
Pub/Sub ( pubsub.googleapis.com/projects/[PROJECT_ID]/topics/[TOPIC_NAME] ):
Primary Use Case: Real-time event streaming to third-party Security Information and Event Management (SIEM) systems (such as Splunk, Datadog, Chronicle, or Elastic) and event-driven automation via Cloud Functions.
Operational Profile: Streaming delivery with variable latency of individual log message payloads.
Another Google Cloud Project: Centralizing logs across hundreds of spoke projects into a dedicated centralized security logging project.
Cloud Logging Log Bucket: Dedicated log buckets with customized retention periods (up to 3,650 days) and regional residency constraints.
Creating Sinks via the gcloud CLI
When a sink is created, Cloud Logging generates a dedicated service account (writer identity) for that sink. You must grant this service account appropriate IAM write permissions on the destination resource: ## Create a sink routing critical errors to Pub/Sub
- gcloud logging sinks create critical-errors-sink \
- pubsub.googleapis.com/projects/my-prod-project/topics/security-alerts \
- -log-filter="severity >= CRITICAL"
## Output displays: writerIdentity: serviceAccount:p12345678-9012@gcp-sa-logging.iam.gserviceaccount.com
- ## Grant the sink service account permission to publish to the target Pub/Sub topic
- gcloud pubsub topics add-iam-policy-binding security-alerts \
- -member="serviceAccount:p12345678-9012@gcp-sa-logging.iam.gserviceaccount.com" \
- -role="roles/pubsub.publisher"
- Log Exclusion Rules & Cost Management
Cloud Logging bills based on log ingestion volume beyond a monthly free tier (first 50 GiB per project). In high-throughput microservices, health-check endpoints, load balancer probes, and debug logging can generate terabytes of benign telemetry, causing runaway cloud bills.
Mechanics of Log Exclusions
Exclusion Filters: Define criteria to prevent matched logs from being ingested into a log bucket ( _Default or custom bucket).
Percentile Exclusion (Sampling): Exclusion rules can drop a percentage of matching logs (e.g., exclude 99% of successful HTTP 200 health check logs while retaining 1% for statistical baseline tracking).
Sink Decoupling: Sinks route logs independently of bucket exclusions. If you create an export sink to BigQuery for all application logs, and configure an exclusion filter on the _Default bucket for debug logs, the debug logs are still delivered to BigQuery while incurring zero Cloud Logging bucket storage costs.
- Example: Excluding Health Check Telemetry ## Exclude 100% of successful HTTP 200 GET requests to /healthz
- gcloud logging sinks update _Default \
--add-exclusion="name=drop-health-probes,filter=resource.type=\"cloud_run_revision\" AND httpRequest.requestUrl:\"/healthz\" AND httpRequest.status=200,description=Drop benign health check logs"
- Architectural Scenarios and Exam Traps
- Scenario A: End-to-End Log and Trace Correlation
Problem: An operations team uses Cloud Trace to detect latency spikes. When clicking on an anomalous span, the "Logs" tab in the Trace details pane displays "No logs found for this trace," even though the service logged extensive details.
Root Cause: The application populated logging.googleapis.com/trace with only the raw 32-character hexadecimal trace ID ( 4bf92f3577b34da6a3ce929d0e0e4736 ).
Resolution: Update the logging library to format the trace field with the full resource string: projects/[PROJECT_ID]/traces/[TRACE_ID] . Cloud Logging requires this full path to establish the relational link in Cloud Trace.
Scenario B: Multi-Region Compliance Audit Archive
Requirement: Regulators mandate that all administrative access and data mutation logs be preserved in an immutable, tamper-proof state for 7 years at the lowest achievable storage cost. Real-time querying is not required.
Resolution: Create a Log Router sink with the filter logName:"cloudaudit.googleapis.com" . Route the sink to a Cloud Storage bucket configured with Archive Storage Class and a Bucket Lock (Object Retention) policy enforcing a 7-year retention period.
Scenario C: Preventing Health Check Ingestion Overruns
Exam Trap: Developers often attempt to suppress health check logs by adding application-level logic to write health logs to a local disk file instead of stdout . Cloud Run's writable container filesystem is memory-backed; ordinary GKE container filesystems are not universally tmpfs. Local files can still exhaust memory or ephemeral storage and are not a durable logging system. On Cloud Run, unbounded writes consume RAM, eventually causing an Out-Of-Memory ( OOMKilled ) container crash. The correct Google Cloud architectural solution is writing to stdout and applying a Log Exclusion Filter at the Log Router.