Study guide

5.2 Cluster Administration: Node Maintenance, Upgrades & etcd Backup

Kubernetes and Cloud Native Associate KCNA

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & Exam Overview

1.1 About the KCNA Exam & Certification Path 1.2 How to Use This Study Guide

2 Chapter 2: Kubernetes Fundamentals - Cluster Architecture & the API

2.1 Kubernetes Architecture & Control Plane Components 2.2 Worker Node Architecture & Node Components 2.3 Kubernetes API, Declarative Management & kubectl 2.4 Kubernetes Releases, API Versioning & Deprecation Policy

3 Chapter 3: Kubernetes Fundamentals - Workloads & Configuration

3.1 Core Workload Objects (Pods, Deployments & ReplicaSets) 3.2 Advanced Controllers (StatefulSets, DaemonSets, Jobs & CronJobs) 3.3 Namespaces, Labels, Selectors & Annotations 3.4 ConfigMaps, Secrets & Application Configuration 3.5 Pod & Container Lifecycle, Restart Policies & Graceful Termination

4 Chapter 4: Kubernetes Fundamentals - Scheduling & Autoscaling

4.1 Scheduling, Resource Management & Node Selection 4.2 Advanced Scheduling: Affinity, Topology Spread, Priority & Disruption Budgets 4.3 Autoscaling Workloads & Clusters (HPA, VPA, Cluster Autoscaler, KEDA)

5 Chapter 5: Kubernetes Fundamentals - Cluster Administration & Extensibility

5.1 Kubernetes Distributions, Installation Tools & Managed Services 5.2 Cluster Administration: Node Maintenance, Upgrades & etcd Backup 5.3 Extending Kubernetes: CRDs, Operators & the Controller Pattern

6 Chapter 6: Kubernetes Fundamentals - Containerization

6.1 Container Fundamentals, Runtimes & Registries 6.2 Container Image Anatomy: Layers, Tags, Digests & Pull Policy

7 Chapter 7: Container Orchestration - Networking & Service Discovery

7.1 Kubernetes Networking, Services & Ingress 7.2 Cluster DNS & Service Discovery with CoreDNS 7.3 NetworkPolicies, Zero-Trust Segmentation & the Gateway API

8 Chapter 8: Container Orchestration - Storage

8.1 Storage Concepts (PVs, PVCs, StorageClasses & CSI) 8.2 Volume Types, Snapshots & Stateful Storage Operations

9 Chapter 9: Container Orchestration - Security

9.1 Kubernetes Security & Access Control (RBAC & ServiceAccounts) 9.2 The 4C's of Cloud Native Security & Supply-Chain Trust 9.3 Service Mesh, mTLS & Zero-Trust Service Identity

10 Chapter 10: Container Orchestration - Troubleshooting

10.1 Container Troubleshooting & Health Probes 10.2 Cluster & Node-Level Troubleshooting

11 Chapter 11: Cloud Native Application Delivery

11.1 Cloud Native CI/CD Pipelines & Build Tooling 11.2 GitOps & Package Management (Helm, Kustomize, Argo CD & Flux) 11.3 Deployment Strategies & Release Management 11.4 Progressive Delivery Automation: Argo Rollouts, Flagger & Feature Flags 11.5 Application Debugging & Developer Workflows in Kubernetes 11.6 Platform Engineering & Developer Self-Service

12 Chapter 12: Cloud Native Architecture - Observability & Principles

12.1 Cloud Native Observability (Logging, Metrics & Tracing) 12.2 Cloud Native Principles & Architecture Patterns 12.3 Serverless, FaaS & Event-Driven Architecture

13 Chapter 13: Cloud Native Architecture - CNCF Ecosystem & Community

13.1 The CNCF Landscape, Governance & Project Lifecycle 13.2 Cloud Native Community, Collaboration & Contribution

5.2 Cluster Administration: Node Maintenance, Upgrades & etcd Backup

Key Takeaways Explain This Topic AI

kubectl cordon marks a node unschedulable without moving anything; kubectl drain cordons the node and evicts its Pods so maintenance can proceed safely.

Cluster upgrades go control plane first, then worker nodes, one minor version at a time — kubelet may lag the API server by up to three minor versions but must never be ahead.

An etcd snapshot is the only complete backup of cluster state, because the API server stores every object there; snapshots are taken with etcdctl snapshot save.

Drain respects PodDisruptionBudgets, which is what converts a rolling node upgrade from an outage into a non-event.

Audit logging, ResourceQuotas, LimitRanges, and namespace-scoped RBAC are the four controls that make a shared cluster safely multi-tenant.

Last updated: August 2026

5.2 Cluster Administration: Node Maintenance, Upgrades & etcd Backup

Quick Answer: Taking a node out of service is a two-step ritual: kubectl cordon stops new Pods landing on it, and kubectl drain evicts what is already there while respecting PodDisruptionBudgets. Cluster upgrades run control plane first, then worker nodes, one minor version at a time . Because kube-apiserver persists every object in etcd , an etcd snapshot is the only complete backup of cluster state.

The official curriculum lists Administration as one of the four competencies under Kubernetes Fundamentals — the largest domain on the exam. KCNA does not ask you to run these commands, but it does ask what they do and in what order.

  • 1. Taking a Node Out of Service # 1. Stop the scheduler placing anything new here
  • kubectl cordon node-07
  • # 2. Evict what is already running
  • kubectl drain node-07 --ignore-daemonsets --delete-emptydir-data

# ... perform kernel patch / hardware swap / kubelet upgrade ...

  • # 3. Return the node to the pool
  • kubectl uncordon node-07
  • Command Effect on new Pods Effect on running Pods
  • cordon Blocked — node marked SchedulingDisabled Untouched, keep running
  • drain Blocked (drain cordons first) Evicted via the Eviction API
  • uncordon Allowed again N/A
  • Why the Two Flags Are Almost Always Needed

--ignore-daemonsets — DaemonSet Pods are immediately recreated on the same node by their controller, so drain refuses to proceed unless told to skip them. Almost every real cluster runs CNI, kube-proxy, and log-shipper DaemonSets, so this flag is effectively mandatory.

--delete-emptydir-data — confirms you accept that data in emptyDir volumes is destroyed. Drain will not silently discard it.

--force — required for naked Pods (Pods with no owning controller), because nothing will recreate them elsewhere. Reach for it knowingly.

Drain Uses the Eviction API, Not Delete

This is the detail that matters. drain issues Eviction requests, which are checked against PodDisruptionBudgets . If evicting a Pod would breach a budget, the request is rejected and drain retries until a replacement becomes Ready elsewhere. That interlock is exactly what turns a node-by-node cluster upgrade into a non-event instead of an outage — and it is why a PDB with no headroom ( minAvailable equal to replicas ) blocks drains forever.

2. Upgrading a Cluster

Kubernetes enforces a bounded version skew , so upgrades follow a fixed order and a fixed step size. STEP 1 Control plane node 1: upgrade kubeadm → apply → upgrade kubelet + kubectl

STEP 2 Remaining control plane nodes, one at a time

STEP 3 Worker nodes, one at a time:

  • cordon → drain → upgrade kubelet/kube-proxy → uncordon
  • STEP 4 Verify: kubectl get nodes shows the new version everywhere

The governing rules:

Never skip a minor version. 1.31 → 1.33 is unsupported; go 1.31 → 1.32 → 1.33.

kubelet may be up to three minor versions older than kube-apiserver , never newer. A node running a newer kubelet than the API server is an unsupported configuration.

Upgrade the control plane before the nodes , so the API server is always the most current component.

Check the deprecation notes first. Removals land at minor releases, so an upgrade is a migration event. Alert on the apiserver_requested_deprecated_apis metric before you begin.

On a managed service the control plane step is a provider action (a console click or API call); the node-pool step remains yours.

3. etcd Backup and Restore

Every object — Deployments, Secrets, RBAC bindings, CRD instances — lives in etcd , and kube-apiserver is the only component that writes to it. Therefore:

  • An etcd snapshot is a complete backup of cluster state. Nothing else is. # Take a snapshot
  • ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-2026-08-07.db \
  • -endpoints=https://127.0.0.1:2379 \
  • -cacert=/etc/kubernetes/pki/etcd/ca.crt \
  • -cert=/etc/kubernetes/pki/etcd/server.crt \
  • -key=/etc/kubernetes/pki/etcd/server.key
  • # Inspect it
  • etcdctl snapshot status /backup/etcd-2026-08-07.db --write-out=table
  • # Restore into a fresh data directory
  • etcdctl snapshot restore /backup/etcd-2026-08-07.db \
  • -data-dir=/var/lib/etcd-restored

Operational points KCNA cares about:

A snapshot captures API objects only . It does not capture the contents of PersistentVolumes — application data needs its own backup, typically via CSI volume snapshots or a tool such as Velero .

Velero is the common ecosystem answer for backing up namespaces plus their PersistentVolumes together, and for cluster migration.

Store snapshots off the control plane node . A backup that dies with the machine is not a backup.

Restoring rolls the whole cluster back to the snapshot instant: objects created since then are gone. Restore is a last resort, not a routine.

4. Certificates

Kubernetes is mutually TLS-authenticated end to end. kubeadm issues client and serving certificates with a one-year lifetime and renews them automatically on each kubeadm upgrade . Clusters that are not upgraded for over a year hit expired certificates and lock everyone out — a genuinely common outage. kubeadm certs check-expiration # what expires when

kubeadm certs renew all # renew, then restart control plane pods

The cluster CA certificate has a ten-year default lifetime and is not rotated by a normal upgrade.

5. Governance Controls for a Shared Cluster

Four mechanisms together make a multi-tenant cluster administrable:

  • Control What it constrains Scope
  • RBAC (Role/RoleBinding) Who may perform which verbs on which resources Namespace or cluster
  • ResourceQuota Aggregate CPU, memory, storage, and object counts a namespace may consume Namespace
  • LimitRange Default, minimum, and maximum requests/limits per container Namespace
  • Audit logging An immutable record of every request the API server handled Cluster
  • Audit Logging

The API server can emit a structured audit event for each request at one of four levels — None , Metadata (who, what, when, verdict), Request (adds the request body), and RequestResponse (adds the response body). An AuditPolicy maps rules to levels, so you can log Secret access at Metadata while logging routine get pods at None . Audit logs are the primary forensic evidence after an incident and are typically shipped off-cluster immediately.

Node Health

Routine administration also means watching node conditions — Ready , MemoryPressure , DiskPressure , PIDPressure — since the kubelet begins evicting Pods under pressure long before the node goes NotReady . A node that oscillates between Ready and NotReady is usually reporting a kubelet, container-runtime, or network-plugin problem rather than a workload problem.