Study guide
6.8 Metrics Server, Resource Utilization & Kubectl Top
Certified Kubernetes Administrator CKA
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: CKA Exam Environment & Cluster Architecture
1.1 CKA Exam Mechanics, Scoring & Remote Terminal Environment 1.2 Kubernetes Control Plane & Worker Node Components 1.3 etcd Architecture, Clustering & Raft Consensus 1.4 Initializing Clusters with Kubeadm 1.5 Highly Available (HA) Control Plane Topologies 1.6 Cluster Lifecycle & Step-by-Step Upgrades 1.7 etcd Snapshot Backup and Restore Procedures
2 Chapter 2: Cluster Extensions, Packaging & RBAC
2.1 Kubernetes TLS Certificates & PKI Infrastructure 2.2 Managing Kubeconfig Files, Users, & Contexts 2.3 Installing Cluster Components with Helm & Kustomize 2.4 Role-Based Access Control (RBAC): Roles & RoleBindings 2.5 ClusterRoles & ClusterRoleBindings 2.6 Custom Resource Definitions (CRDs) & Operators 2.7 Pod Security Standards (PSS) & Admission Controllers
3 Chapter 3: Workloads, Lifecycle & Pod Scheduling
3.1 Pod Lifecycle, Multi-Container Patterns & Init Containers 3.2 Deployments: Rolling Updates, Rollouts & Rollbacks 3.3 DaemonSets & StatefulSets Architecture 3.4 Batch Workloads: Jobs & CronJobs 3.5 Managing ConfigMaps & Sensitive Data with Secrets 3.6 Resource Requests, Limits & Namespace ResourceQuotas 3.7 NodeSelectors, Node Affinity & Anti-Affinity 3.8 Node Taints & Pod Tolerations 3.9 Autoscaling: Horizontal Pod Autoscaler (HPA) & Custom Metrics
4 Chapter 4: Services, Networking & Ingress Architecture
4.1 Kubernetes IP-per-Pod Model & CNI Plugins (Flannel, Calico, Cilium) 4.2 Services Deep Dive: ClusterIP, NodePort, LoadBalancer & EndpointSlices 4.3 Kube-Proxy Service Dataplane: iptables, nftables, IPVS & eBPF 4.4 CoreDNS Configuration, Resolution & Custom Forwarding 4.5 NetworkPolicies: Ingress, Egress & Default-Deny Isolation 4.6 Ingress Architecture, Controllers & Path-Based Routing 4.7 Gateway API: GatewayClass, Gateway, HTTPRoute & Traffic Splitting 4.8 External Traffic Ingress: NodePort, Cloud LoadBalancers & MetalLB
5 Chapter 5: Storage Architecture & Persistent Volumes
5.1 Ephemeral Volumes, EmptyDir, HostPath & Config Volumes 5.2 PersistentVolumes (PV) & PersistentVolumeClaims (PVC) Lifecycle 5.3 StorageClasses, Dynamic Provisioning & VolumeBindingModes 5.4 Storage Access Modes, Reclaim Policies & Volume Expansion 5.5 Container Storage Interface (CSI) Architecture & Drivers 5.6 StatefulSet VolumeClaimTemplates & Persistent Storage Workflows
6 Chapter 6: Advanced Troubleshooting, Monitoring & Maintenance
6.1 Systematic Kubernetes Troubleshooting Methodology 6.2 Node Failure, Kubelet Diagnostics & Systemd Services 6.3 Control Plane Failures, Static Pods & Component Logs 6.4 etcd Troubleshooting, Quorum Loss & Database Defragmentation 6.5 Pod Startup Failures, CrashLoopBackOff & OOMKilled Diagnostics 6.6 Workload Scheduling Failures & Resource Constraint Resolution 6.7 Container & Node Logging Architecture, Journalctl & Fluentd 6.8 Metrics Server, Resource Utilization & Kubectl Top 6.9 Service, Endpoint & Ingress Connectivity Troubleshooting 6.10 Storage Failures: PVC Pending, Mount Failures & Multi-Attach Errors 6.11 NetworkPolicy Isolation & CoreDNS Resolution Diagnostics
6.8 Metrics Server, Resource Utilization & Kubectl Top
Key Takeaways Explain This Topic AI
Metrics Server is an in-memory resource metrics pipeline that collects CPU and memory data from each kubelet metrics endpoint and exposes it through the aggregated metrics.k8s.io API.
Metrics Server does NOT store historical metrics and is not a replacement for full monitoring solutions like Prometheus; its primary role is powering 'kubectl top' and driving Horizontal Pod Autoscalers (HPA).
The 'kubectl top nodes' and 'kubectl top pods' commands display live CPU (cores/millicores) and Memory (bytes/MiB) utilization, supporting sorting with '--sort-by=cpu' and '--sort-by=memory'.
Common failures include kubelet serving-certificate trust and node address resolution. Prefer valid CA-signed kubelet certificates and reachable InternalIP addresses; --kubelet-insecure-tls is only a temporary test workaround.
Production monitoring relies on the Four Golden Signals (Latency, Traffic, Errors, Saturation) and the USE/RED methods implemented via Prometheus, Alertmanager, and Grafana.
Last updated: August 2026
6.8 Metrics Server, Resource Utilization & Kubectl Top
Resource observability is essential for capacity planning, detecting runaway memory leaks, preventing CPU starvation, and autoscaling workloads. Kubernetes decouples short-term operational resource metrics from full long-term historical time-series monitoring.
The Metrics Server provides the foundational pipeline for core Kubernetes autoscaling and CLI resource inspection ( kubectl top ), while tools like Prometheus and Grafana provide deep observability, alerting, and trend analysis.
1. Metrics Server Architecture & API Aggregation
Metrics Server is an add-on component that operates as an aggregated API server. It periodically (default resolution: 15 seconds) collects container and node CPU/memory telemetry from each kubelet's resource-metrics endpoint; kubelet obtains container measurements from its integrated cAdvisor instrumentation across every node in the cluster. +-----------------------------------------------------------------------------------------+
- | METRICS SERVER PIPELINE ARCHITECTURE |
- | |
- | [KUBECTL TOP / HPA] |
- | | |
- | v (HTTPS: Query /apis/metrics.k8s.io/v1beta1/) |
- | [KUBE-APISERVER (API Aggregator Proxy)] |
- | | |
- | v (Proxies request to extension API server) |
- | [METRICS SERVER POD (kube-system)] |
- | - In-memory ephemeral metric cache (No persistent database) |
- | - Collects metrics over HTTPS (port 10250) every 15s by default |
- | | |
- | +-----------------------+-----------------------+ |
- | | | | |
- | v v v |
- | [NODE 01: KUBELET] [NODE 02: KUBELET] [NODE 03: KUBELET] |
- | cAdvisor /stats/summary cAdvisor /stats/summary cAdvisor /stats/summary |
- +-----------------------------------------------------------------------------------------+
Core Functional Characteristics:
In-Memory Storage: Metrics Server does not persist data to disk or etcd. It retains only the most recent scrape in RAM.
API Endpoint Registration: It registers the API group metrics.k8s.io/v1beta1 with kube-apiserver via an APIService object.
Consumers: Powers kubectl top nodes , kubectl top pods , and the Horizontal Pod Autoscaler (HPA) / Vertical Pod Autoscaler (VPA) .
2. Inspecting Resource Utilization with kubectl top
Once Metrics Server is functional, administrators can query real-time compute consumption: # 1. Inspect resource utilization across all cluster nodes
kubectl top nodes
# Example Output:
- # NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
- # control-plane 250m 12% 1850Mi 48%
- # worker-1 850m 42% 3200Mi 82%
- # worker-2 120m 6% 980Mi 25%
- # 2. Inspect resource utilization for pods in default namespace
- kubectl top pods
- # 3. Query all pods across all namespaces sorted by memory consumption
- kubectl top pods -A --sort-by=memory
- # 4. Query all pods across all namespaces sorted by CPU consumption
- kubectl top pods -A --sort-by=cpu
- # 5. Display individual container metrics within multi-container pods
- kubectl top pods my-app-pod --containers
Metric Units Explained:
CPU: Measured in millicores ( m m m ) or fractions of a vCPU/core ( 1000 m = 1 vCPU / Core 1000m = 1 \text{ vCPU / Core} 1000 m = 1 vCPU / Core ). 250 m = 0.25 CPU cores 250m = 0.25 \text{ CPU cores} 250 m = 0.25 CPU cores .
Memory: Measured in bytes or Megabytes/Gigabytes ( M i Mi M i = Mebibytes = 2 20 = 2^{20} = 2 20 bytes; G i Gi G i = Gibibytes = 2 30 = 2^{30} = 2 30 bytes).
3. High-Yield Metrics Server Troubleshooting
On the CKA exam and fresh cluster installations, kubectl top frequently fails with: error: Metrics API not available or error: metrics not available yet . +-----------------------------------------------------------------------------------------+
- | METRICS SERVER TROUBLESHOOTING DECISION TREE |
- | |
- | $ kubectl top nodes -> FAILS |
- | 1. Check Pod Status: $ kubectl get pods -n kube-system -l k8s-app=metrics-server |
- | - If CrashLoopBackOff: $ kubectl logs -n kube-system deploy/metrics-server |
- | |
- | 2. Common Error A: 'x509: certificate signed by unknown authority' |
- | - Cause: Kubelet serving cert is self-signed; Metrics Server TLS verification fails.|
| - Production fix: issue kubelet serving certificates trusted by Metrics Server. For a disposable test cluster only, `--kubelet-insecure-tls` can confirm that certificate validation is the blocker. |
- | |
- | 3. Common Error B: 'dial tcp: lookup node01 on 10.96.0.10:53: no such host' |
- | - Cause: Metrics Server trying to reach nodes by hostname without cluster DNS records|
- | - Fix: Add `--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname` |
- +-----------------------------------------------------------------------------------------+
Inspecting Metrics Server Arguments:
Edit the deployment only after identifying the failing address or certificate path: kubectl edit deployment metrics-server -n kube-system
Update the container command-line arguments: spec:
template:
spec:
containers:
- name: metrics-server
args:
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --metric-resolution=15s
# Test-only diagnostic workaround; prefer trusted kubelet serving certs:
- # - --kubelet-insecure-tls
- 4. Full Monitoring Stack & The Four Golden Signals
For enterprise production clusters, Metrics Server is augmented with Prometheus (pull-based metrics collector and time-series database) and Grafana (visualization dashboards). +-----------------------------------------------------------------------------------------+
- | THE FOUR GOLDEN SIGNALS (SRE) |
- | |
- | 1. LATENCY Time taken to service a request (e.g., HTTP 99th percentile < 200ms). |
- | 2. TRAFFIC Demand placed on system (e.g., HTTP requests/sec, IOPS, network bps). |
- | 3. ERRORS Rate of requests that fail (e.g., HTTP 5xx responses, error logs). |
- | 4. SATURATION Fraction of resource capacity utilized (e.g., memory, CPU, disk IO). |
- +-----------------------------------------------------------------------------------------+
Complementary Frameworks:
USE Method (Node/Infrastructure Focus): U tilization (percentage busy), S aturation (queued work), E rrors (hardware/kernel errors).
RED Method (Microservices Focus): R ate (requests per second), E rrors (failed requests), D uration (request latency).