Study guide

5.4 Network I/O Control (NIOC) & TCP/IP Stacks

VMware Certified Professional - Data Center Virtualization VCP-DCV

Track your progress as you read

Finished sections get a check mark.

1 Introduction: VCP-DCV Exam Overview

About the VCP-DCV Exam (2V0-21.23)

2 Chapter 1: vSphere Architecture & Core Platform Technologies

1.1 vSphere 8.x Architecture and Components Overview 1.2 vCenter Server Appliance (vCSA) Architecture & Deployment 1.3 vSphere Distributed Services Engine (DSE) & DPUs 1.4 vSphere Products, Solutions & Workload Platform 1.5 vCenter Topology: SSO Domains & Enhanced Linked Mode 1.6 vSphere with Tanzu: Supervisor, Namespaces, Zones & Lifecycle 1.7 vSphere in the SDDC: vCenter Converter, vSphere Replication & SRM

3 Chapter 2: Compute Resources & Cluster Management

2.1 vSphere High Availability (HA) Architecture 2.2 Distributed Resource Scheduler (DRS) & Resource Pools 2.3 vMotion & Enhanced vMotion Compatibility 2.4 Virtual Machine Advanced Compute Configuration 2.5 Fault Tolerance, Proactive HA & Advanced Cluster Options 2.6 vSphere Cluster Services (vCLS) & Retreat Mode 2.7 Migration Types: Cold Migration, Storage vMotion & Cross vCenter Export

4 Chapter 3: Storage Technologies & VMFS Datastores

3.1 Storage Access Protocols & Multipathing 3.2 VMFS Datastore Architecture & Management 3.3 Storage Policies (SPBM) & Virtual Volumes (vVols) 3.4 Storage DRS & Storage I/O Control 3.5 RDM, PMem, NVMe, NVMe-oF & iSER Use Cases

5 Chapter 4: VMware vSAN Architecture & Operations

4.1 vSAN Core Architecture & Disk Groups 4.2 vSAN Fault Domains & Stretched Clusters 4.3 vSAN Storage Policies & Data Protection 4.4 vSAN Maintenance, Health Checks & Monitoring

6 Chapter 5: Virtual Networking Architecture & Configuration

5.1 Standard Switches (vSS) vs Distributed Switches (vDS) 5.2 Port Group Policies & NIC Teaming 5.3 VLAN Configuration & Security Policies 5.4 Network I/O Control (NIOC) & TCP/IP Stacks

7 Chapter 6: Security, Identity & Access Control

6.1 vCenter Single Sign-On (SSO) & Identity Sources 6.2 Role-Based Access Control (RBAC) & Global Permissions 6.3 Virtual Machine Encryption & Key Management 6.4 ESXi Host Security & vSphere Trust Authority 6.5 vSphere Certificate Management 6.6 VM Firmware, Secure Boot, vTPM & Virtualization-Based Security

8 Chapter 7: Lifecycle Management & Host Configuration

7.1 vSphere Lifecycle Manager (vLCM) Cluster Images 7.2 Baseline-Based Lifecycle Management 7.3 Host Profiles & Cluster Quickstart 7.4 Content Libraries & Virtual Machine Deployment 7.5 ESXi Host Configuration: Time, Services, Product Locker & Boot Options 7.6 vCenter Lifecycle: Update Planner, VAMI Patching & Upgrades 7.7 Virtual Machine Snapshots & VM Creation Methods

9 Chapter 8: Monitoring, Performance Tuning & Troubleshooting

8.1 Performance Monitoring & Metrics 8.2 Command-Line Diagnostics & esxtop 8.3 Log Analysis & Diagnostic Data Collection 8.4 Troubleshooting Common Infrastructure Scenarios 8.5 Alarms & vCenter Appliance Health Monitoring

5.4 Network I/O Control (NIOC) & TCP/IP Stacks

Key Takeaways Explain This Topic AI

Network I/O Control version 3 (NIOC v3) introduces cluster-wide bandwidth reservations for system traffic types and user-defined VM network resource pools.

NIOC bandwidth allocation utilizes Shares (relative weight during congestion in a 1:2:4 ratio), Reservations (guaranteed minimum bandwidth up to 75% of physical uplink capacity), and Limits (hard ceiling).

The dedicated vMotion TCP/IP stack provides an independent routing table, default gateway, and ARP cache, enabling Layer-3 routed vMotion across disparate subnets without routing through the management gateway.

The Provisioning TCP/IP stack isolates cold virtual machine migrations, cloning, and snapshot traffic from production vMotion and management traffic.

Custom TCP/IP stacks can be created via ESXCLI ( esxcli network ip netstack add ) to satisfy compliance, multitenancy, or isolated storage routing requirements.

Last updated: September 2026

5.4 Network I/O Control (NIOC) & TCP/IP Stacks

Modern data center architectures converge multiple disparate network traffic types—such as Management, vMotion, vSAN, iSCSI, and virtual machine workloads—onto high-bandwidth physical network interfaces (10GbE, 25GbE, and 100GbE). To prevent high-volume burst traffic from starving latency-sensitive services, vSphere provides Network I/O Control (NIOC) . Complementing NIOC, vSphere provides Dedicated TCP/IP Stacks to separate routing domains and default gateways across host services.

Network I/O Control version 3 (NIOC v3) Architecture

Network I/O Control version 3 is an enterprise QoS mechanism available exclusively on vSphere Distributed Switches. (A vDS created with Network Offloads compatibility for DPUs has NIOC disabled; see Section 1.3.) In converged networking designs where distinct traffic flows share common physical uplinks, an unconstrained operation (such as a massive parallel vMotion migration or storage array backup) can saturate physical link buffers and induce severe packet drops for vSAN storage or management heartbeats.

NIOC v3 guarantees predictable quality of service by enforcing bandwidth scheduling directly at the vDS transmission queue layer.

System Traffic Types Managed by NIOC

NIOC v3 recognizes and independently schedules nine predefined system traffic types:

vSphere vMotion: Live virtual machine state and memory transfer.

Management Traffic: Host-to-vCenter communication, HA heartbeats, and ESXCLI/API calls.

vSAN Traffic: Distributed storage synchronization, read/write I/O, and witness heartbeats.

Fault Tolerance (FT) Logging: Lockstep execution synchronization between FT VM pairs.

iSCSI Traffic: Software and hardware iSCSI block storage protocols.

NFS Traffic: Network File System datastore storage I/O.

vSphere Replication (VR): Asynchronous replication change streams.

vSphere Backup (NFC): Network File Copy traffic utilized by backup appliances.

Virtual Machine Traffic: Production and non-production guest OS traffic flows.

Bandwidth Allocation Primitives: Shares, Reservations, and Limits

NIOC v3 controls traffic through three parameters: +---------------------------------------------------------------------------------------------------+

  • | NIOC v3 Bandwidth Primitives |
  • +-----------------+--------------------------+-----------------------+------------------------------+
  • | Primitive | Unit of Measurement | When Enforced? | Operational Impact |
  • +-----------------+--------------------------+-----------------------+------------------------------+
  • | Shares | Low (25) / Normal (50) | Only during physical | Relative ratio of excess |
  • | | High (100) / Custom | link congestion | bandwidth distribution |
  • +-----------------+--------------------------+-----------------------+------------------------------+
  • | Reservations | Megabits per second | Constantly guaranteed | Minimum guaranteed bandwidth;|
  • | | (Mbps) or % of link | even during failure | subject to 75% max cap |
  • +-----------------+--------------------------+-----------------------+------------------------------+
  • | Limits | Megabits per second | Constantly enforced | Hard throughput ceiling; |
  • | | (Mbps) or Gbps | at all times | cannot burst past limit |
  • +-----------------+--------------------------+-----------------------+------------------------------+
  • 1. Shares

Mechanism: Shares define the relative priority of a traffic type only when a physical uplink experiences congestion (100% saturation) . Under uncongested conditions, any traffic type can consume up to 100% of the available uplink bandwidth.

Predefined Ratios: Shares follow a standard 1:2:4 ratio :

  • Low: 25 shares
  • Normal: 50 shares
  • High: 100 shares
  • Custom: Explicit numerical value

Contention Math: If vSAN traffic is set to High (100 shares) and vMotion traffic is set to Normal (50 shares), and both contend for a saturated 10GbE uplink, vSAN is guaranteed 100 / (100 + 50) = 66.7% of the link capacity (6.67 Gbps), while vMotion receives 50 / (100 + 50) = 33.3% (3.33 Gbps).

2. Reservations

Mechanism: A reservation specifies a guaranteed minimum bandwidth allocation in Megabits per second (Mbps) or as a percentage of physical link capacity. Unlike shares, a reservation is statically guaranteed, ensuring that latency-sensitive traffic (such as vSAN or Management) always receives sufficient throughput even during catastrophic link saturation.

Cluster-Wide Enforcement & Admission Control: In NIOC v3, reservations can be applied cluster-wide. When a virtual machine associated with a network resource pool powers on, vSphere evaluates whether the underlying host has sufficient physical bandwidth reservation capacity. If the reservation cannot be satisfied, the VM power-on operation is blocked by admission control.

The 75% Maximum Reservation Rule (Exam Trap):

NIOC v3 enforces a strict maximum reservation ceiling: The sum of all reservations for system traffic types and VM resource pools cannot exceed 75% of the capacity of the lowest-speed physical uplink in the team .

Example: On an ESXi host with two 10 Gbps uplinks, the maximum allocatable reservation pool is 7.5 Gbps (75% of 10 Gbps). The remaining 25% (2.5 Gbps) is permanently unreserved to accommodate system overhead, burst traffic, and unreserved traffic.

3. Limits

Mechanism: A limit imposes a strict maximum throughput ceiling on a traffic type or VM network resource pool. Even if the physical network adapters are 90% idle, a traffic type capped by a limit cannot exceed its configured threshold.

Use Case: Restricting aggressive non-critical traffic—such as developmental VM replication or backup copying—from consuming excess bandwidth.

User-Defined Virtual Machine Network Resource Pools

In addition to system traffic types, NIOC v3 allows administrators to create custom Virtual Machine Network Resource Pools :

Custom Tiers: Administrators create resource pools (e.g., Production-Tier , Dev-Test-Tier ) and assign shares, reservations, and limits to each pool.

Distributed Port Group Association: Individual distributed port groups are mapped to these network resource pools. All virtual machines connected to that distributed port group automatically draw bandwidth from that pool's quota.

Single Root I/O Virtualization (SR-IOV) Exception: Virtual machines configured with SR-IOV bypass the VMkernel virtual switch entirely, connecting directly to the physical NIC's virtual functions (VF). Consequently, SR-IOV workloads cannot be shaped, monitored, or throttled by NIOC v3.

Dedicated TCP/IP Stacks in vSphere

In early vSphere versions, an ESXi host operated with a single unified TCP/IP stack. All VMkernel adapters ( vmk0 , vmk1 , vmk2 ) shared a single routing table, a single DNS configuration, and a single system default gateway. This created major architectural limitations when vMotion or IP storage traffic needed to be routed across Layer-3 network boundaries without traversing the management gateway.

Modern vSphere implements Dedicated TCP/IP Stacks to provide complete routing isolation. +-------------------------------------------------------------------------+

  • | ESXi Host VMkernel Stacks |
  • +--------------------+---------------------+------------------------------+
  • | Default Stack | vMotion Stack | Provisioning Stack |
  • | - Routing Table A | - Routing Table B | - Routing Table C |
  • | - Gateway: vmk0 | - Gateway: vmk1 | - Gateway: vmk2 |
  • | - DNS: Corporate | - No DNS needed | - Optional DNS |
  • | - Mgmt, HA, Syslog | - vMotion Only | - Cold VM clone, snapshot |
  • +--------------------+---------------------+------------------------------+
  • Built-in System TCP/IP Stacks

Every ESXi host includes three pre-configured TCP/IP stacks:

Default TCP/IP Stack:

Handles host management traffic ( vmk0 ), DNS lookups, Active Directory authentication, syslog forwarding, SNMP, NTP, and any VMkernel interface not explicitly bound to a dedicated stack.

Uses the host's system default gateway.

vMotion TCP/IP Stack:

Exclusively handles vSphere vMotion traffic.

Features its own isolated routing table, default gateway, and ARP cache.

Primary Benefit: Enables Layer-3 routed vMotion across data centers or routed campus clusters without requiring stretched Layer-2 VLANs, complex static routes, or management gateway traversal.

Provisioning TCP/IP Stack:

Carries provisioning traffic, which uses Network File Copy (NFC): cold migration of powered-off VMs, cloning, and snapshot migration. It can also carry the NFC portion of long-distance vMotion.

Isolates bulk image transfer traffic from production vMotion and management networks.

Custom TCP/IP Stacks

For specialized networking requirements—such as multi-tenant isolation, dedicated backup networks, or routed iSCSI/NFS storage arrays—administrators can define custom TCP/IP stacks:

Creation via CLI: Custom stacks are created via the ESXi command line: esxcli network ip netstack add -N BackupStack

Configuration: Once created, administrators can assign a dedicated default gateway, custom DNS servers, and specific MTU settings to the custom stack.

VMkernel Binding: A new VMkernel adapter is created and bound directly to the custom TCP/IP stack, providing complete routing table isolation from the default management network.

Loading diagram... NIOC v3 Traffic Scheduling and Dedicated TCP/IP Stacks