ExamFinch.com My Study

Study guide

11.5 Application Debugging & Developer Workflows in Kubernetes

Practice this exam

Kubernetes and Cloud Native Associate KCNA

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & Exam Overview

1.1 About the KCNA Exam & Certification Path 1.2 How to Use This Study Guide

2 Chapter 2: Kubernetes Fundamentals - Cluster Architecture & the API

2.1 Kubernetes Architecture & Control Plane Components 2.2 Worker Node Architecture & Node Components 2.3 Kubernetes API, Declarative Management & kubectl 2.4 Kubernetes Releases, API Versioning & Deprecation Policy

3 Chapter 3: Kubernetes Fundamentals - Workloads & Configuration

3.1 Core Workload Objects (Pods, Deployments & ReplicaSets) 3.2 Advanced Controllers (StatefulSets, DaemonSets, Jobs & CronJobs) 3.3 Namespaces, Labels, Selectors & Annotations 3.4 ConfigMaps, Secrets & Application Configuration 3.5 Pod & Container Lifecycle, Restart Policies & Graceful Termination

4 Chapter 4: Kubernetes Fundamentals - Scheduling & Autoscaling

4.1 Scheduling, Resource Management & Node Selection 4.2 Advanced Scheduling: Affinity, Topology Spread, Priority & Disruption Budgets 4.3 Autoscaling Workloads & Clusters (HPA, VPA, Cluster Autoscaler, KEDA)

5 Chapter 5: Kubernetes Fundamentals - Cluster Administration & Extensibility

5.1 Kubernetes Distributions, Installation Tools & Managed Services 5.2 Cluster Administration: Node Maintenance, Upgrades & etcd Backup 5.3 Extending Kubernetes: CRDs, Operators & the Controller Pattern

6 Chapter 6: Kubernetes Fundamentals - Containerization

6.1 Container Fundamentals, Runtimes & Registries 6.2 Container Image Anatomy: Layers, Tags, Digests & Pull Policy

7 Chapter 7: Container Orchestration - Networking & Service Discovery

7.1 Kubernetes Networking, Services & Ingress 7.2 Cluster DNS & Service Discovery with CoreDNS 7.3 NetworkPolicies, Zero-Trust Segmentation & the Gateway API

8 Chapter 8: Container Orchestration - Storage

8.1 Storage Concepts (PVs, PVCs, StorageClasses & CSI) 8.2 Volume Types, Snapshots & Stateful Storage Operations

9 Chapter 9: Container Orchestration - Security

9.1 Kubernetes Security & Access Control (RBAC & ServiceAccounts) 9.2 The 4C's of Cloud Native Security & Supply-Chain Trust 9.3 Service Mesh, mTLS & Zero-Trust Service Identity

10 Chapter 10: Container Orchestration - Troubleshooting

10.1 Container Troubleshooting & Health Probes 10.2 Cluster & Node-Level Troubleshooting

11 Chapter 11: Cloud Native Application Delivery

11.1 Cloud Native CI/CD Pipelines & Build Tooling 11.2 GitOps & Package Management (Helm, Kustomize, Argo CD & Flux) 11.3 Deployment Strategies & Release Management 11.4 Progressive Delivery Automation: Argo Rollouts, Flagger & Feature Flags 11.5 Application Debugging & Developer Workflows in Kubernetes 11.6 Platform Engineering & Developer Self-Service

12 Chapter 12: Cloud Native Architecture - Observability & Principles

12.1 Cloud Native Observability (Logging, Metrics & Tracing) 12.2 Cloud Native Principles & Architecture Patterns 12.3 Serverless, FaaS & Event-Driven Architecture

13 Chapter 13: Cloud Native Architecture - CNCF Ecosystem & Community

13.1 The CNCF Landscape, Governance & Project Lifecycle 13.2 Cloud Native Community, Collaboration & Contribution

11.5 Application Debugging & Developer Workflows in Kubernetes

Key Takeaways Explain This Topic AI

kubectl debug attaches an ephemeral container to a running Pod, which is the only way to get a shell alongside a distroless or scratch-based container.

kubectl debug can also create a copy of a Pod with a modified command or image, useful when the original crashes before a shell can attach.

kubectl port-forward tunnels a local port to a Pod or Service so a developer can reach an internal endpoint without exposing it publicly.

kubectl cp copies files between a local machine and a container, and kubectl debug node/<name> opens a privileged debugging Pod on a node host.

Inner-loop tools such as Skaffold, Tilt, Telepresence, and DevSpace shorten the code-build-deploy cycle that otherwise makes Kubernetes development slow.

Last updated: August 2026

11.5 Application Debugging & Developer Workflows in Kubernetes

Quick Answer: The official KCNA curriculum lists Debugging as a competency under Cloud Native Application Delivery. The core tools are kubectl debug (attach an ephemeral container to a running Pod, or clone the Pod with a changed image or command), kubectl port-forward (tunnel a local port to an in-cluster endpoint), kubectl cp (move files in and out), and kubectl debug node/<name> (a privileged Pod on the host). Around them sits the inner-loop tooling — Skaffold, Tilt, Telepresence, DevSpace — that makes day-to-day development bearable.

Section 10.1 covered diagnosing failing containers . This section covers working on an application that is already running.

1. The Distroless Problem

Section 6.1 recommended minimal base images: distroless or scratch , no shell, no package manager, no curl . That is excellent security and terrible ergonomics — kubectl exec -it pod -- /bin/sh fails with executable file not found because there is genuinely no shell in the image.

Ephemeral containers resolve the conflict. kubectl debug adds a temporary container to an already-running Pod , sharing its network namespace and — with --target — its process namespace: kubectl debug -it payments-7d9f8-x2k4 \

  • -image=busybox:1.36 \
  • -target=api \
  • - sh

Inside that ephemeral container you can now:

  • wget -qO- http://localhost:8080/healthz — the app's own port, because the network namespace is shared,
  • nslookup payments.finance.svc.cluster.local — test DNS from exactly this Pod's perspective,
  • ps aux — see the application's processes, because --target shares the process namespace,

inspect /proc/1/environ and mounted volumes.

Properties worth memorising: an ephemeral container cannot be added to a Pod template , has no resource requests or limits , cannot have probes , and is never restarted . It exists to debug one Pod, once.

2. Debugging a Pod That Will Not Start

Ephemeral containers only help a Pod that is running. For a Pod stuck in CrashLoopBackOff , kubectl debug can instead build a copy : # Copy the Pod but replace the entrypoint with a shell,

  • # so the failing command never runs and you can inspect the filesystem
  • kubectl debug payments-7d9f8-x2k4 -it \
  • -copy-to=payments-debug \
  • -container=api \
  • - sh
  • # Copy the Pod but swap in a fatter image that has debugging tools
  • kubectl debug payments-7d9f8-x2k4 \
  • -copy-to=payments-debug \
  • -set-image=api=myapp:v2.1.0-debug

The copy keeps the original's volumes, environment, and ServiceAccount, so configuration and mount problems reproduce faithfully — while the original Pod is left untouched for the record. Delete the copy when finished.

  • 3. Reaching Into the Cluster
  • kubectl port-forward

Tunnels a local port to a Pod or Service through the API server: kubectl port-forward pod/payments-7d9f8 8080:8080

  • kubectl port-forward svc/postgres 5432:5432 -n database
  • kubectl port-forward deploy/grafana 3000:3000 -n monitoring

This is how you open a database GUI against an in-cluster database, or reach an internal admin endpoint, without creating a NodePort or a LoadBalancer . Traffic rides the authenticated API connection, so no ingress path is exposed and your existing RBAC still applies. Note that for a Service it forwards to one backing Pod, not a load-balanced set.

kubectl proxy

Runs an authenticated local proxy to the API itself, useful for exploring raw API endpoints and reaching Service URLs through the API server: kubectl proxy --port=8001

  • curl http://localhost:8001/api/v1/namespaces/production/pods
  • kubectl cp kubectl cp production/payments-7d9f8:/app/heapdump.hprof ./heapdump.hprof
  • kubectl cp ./fixture.json production/payments-7d9f8:/tmp/fixture.json

Pull a heap dump, core file, or log out for offline analysis; push a test fixture in. ( tar must exist in the container, so distroless images need the ephemeral-container route instead.)

4. Debugging the Node

When the problem is beneath the Pod, kubectl debug can open a privileged Pod in the host's namespaces , with the host filesystem mounted at /host : kubectl debug node/worker-07 -it --image=ubuntu

# then, inside:

  • chroot /host
  • journalctl -u kubelet -n 100
  • crictl ps -a
  • df -h

This is the supported alternative to SSH-ing into nodes, and it is exactly the capability that Pod Security Standards restrict — which is why it should be gated by RBAC and audited.

5. Reading Application Failures

Symptom Where to look

Container starts then exits 0 immediately The image's entrypoint completed — a foreground process is required, not a background daemon

exec format error Architecture mismatch: an amd64 image on an ARM node. Build a multi-arch image (section 6.2)

permission denied writing a file readOnlyRootFilesystem: true , or runAsUser does not own the path — add an emptyDir or set fsGroup

  • Env var empty although a ConfigMap exists The key name is wrong, or the ConfigMap is in a different namespace
  • App cannot reach a dependency Test from an ephemeral container: DNS first, then TCP, then HTTP

Works locally, fails in-cluster Almost always configuration, DNS, or a NetworkPolicy — compare the two environments explicitly

A disciplined bottom-up sequence — DNS resolves, TCP connects, HTTP responds — isolates the layer in three commands instead of an afternoon.

6. The Inner Loop

The naive Kubernetes development cycle is edit → build image → push → update manifest → wait for rollout → test , which can take minutes per keystroke-level change. Inner-loop tools compress it:

  • Tool Approach
  • Skaffold Watches source, rebuilds and redeploys automatically, streams logs back; profiles for dev vs prod

Tilt Similar loop with a live web UI showing every service's build and runtime status; supports live file sync into running containers

Telepresence Reroutes traffic for one in-cluster Service to a process on your laptop , so you can use a normal local debugger against real cluster dependencies

  • DevSpace Dev containers with file sync and hot reload
  • kind / minikube A disposable local cluster so the whole loop runs offline

Telepresence deserves the emphasis: it lets a developer set an ordinary breakpoint in their IDE and have real production-shaped traffic hit it, which no amount of log-reading matches.

7. Debugging Etiquette in Shared Clusters

Prefer --copy-to over debugging the live Pod when the workload is serving traffic.

Delete debug copies and ephemeral workloads afterwards — they consume quota and confuse the next person.

Never leave a port-forward running to a production database.

Remember that ephemeral containers and node debug Pods are audited privileged actions , and treat the access accordingly.