Study guide
14.2 Data Deduplication Architecture, Usage Types & Management
Administering Windows Server Hybrid Core Infrastructure AZ-800
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
About Exam AZ-800: Format, Scoring & the September 2026 Retirement
2 Chapter 1: Active Directory Domain Services Architecture & Deployment
1.1 Deploying & Managing Domain Controllers On-Premises 1.2 Domain Controllers in Azure IaaS & Virtualized DC Cloning 1.3 Read-Only Domain Controllers (RODCs) & Password Replication Policies 1.4 Flexible Single Master Operation (FSMO) Roles & Troubleshooting 1.5 Active Directory Sites, Subnets & Site Link Topology 1.6 AD DS Replication: Topology Generation, Convergence & Diagnostics 1.7 Forest and Domain Trust Relationships
3 Chapter 2: AD DS Security Principals, Service Accounts & Domain Join
2.1 Managing Users, Groups & Objects in Multi-Domain and Multi-Forest Environments 2.2 Service Account Types & Group Managed Service Accounts (gMSA) 2.3 Joining Windows Server to AD DS, Microsoft Entra Domain Services & Microsoft Entra ID 2.4 Creating & Managing Users, Groups and Organizational Units at Scale
4 Chapter 3: Hybrid Identity & Microsoft Entra Integration
3.1 Microsoft Entra Connect Sync Architecture, Installation & Filtering 3.2 SourceAnchor, Synchronization Rules & Microsoft Entra Connect Health 3.3 Microsoft Entra Cloud Sync & Disconnected Multi-Forest Topologies 3.4 Hybrid Authentication Methods & Staged Rollout 3.5 Microsoft Entra Domain Services 3.6 Integrating Microsoft Entra ID, AD DS & Entra Domain Services: Choosing an Architecture
5 Chapter 4: Group Policy in AD DS and Hybrid Environments
4.1 Group Policy Architecture & Processing Order (LSDOU) 4.2 Group Policy Filtering, Loopback Processing & Client-Side Extensions 4.3 Group Policy Preferences, Item-Level Targeting & Central Store 4.4 Group Policy in Microsoft Entra Domain Services & Fine-Grained Password Policies
6 Chapter 5: Remote Server Management in Hybrid Environments
5.1 Windows Admin Center (WAC) Deployment On-Premises and in Azure 5.2 PowerShell Remoting, WinRM & the Kerberos Double-Hop Problem 5.3 Just Enough Administration (JEA) for Constrained PowerShell Endpoints 5.4 OpenSSH on Windows Server & PowerShell Remoting over SSH 5.5 Secure RDP Access, Remote Desktop Gateway & Azure Bastion
7 Chapter 6: Hybrid Management with Azure Arc and Azure Services
6.1 Azure Arc-Enabled Servers Architecture & Onboarding 6.2 Azure Machine Configuration & VM Extensions on Arc Servers 6.3 Azure Update Manager: Assessment, Maintenance Configurations & Dynamic Scoping 6.4 Azure Automation for Hybrid Workloads: Runbooks & Hybrid Runbook Workers
8 Chapter 7: Hyper-V Virtualization & Virtual Machine Configuration
7.1 Hyper-V Type 1 Architecture, Installation & Hardware Prerequisites 7.2 VM Generations, Secure Boot, vTPM, Enhanced Session Mode & Direct VM Access 7.3 Compute, Dynamic Memory & Hardware Acceleration 7.4 VM Resource Controls, CPU Groups & Host Compute Partitioning 7.5 Hyper-V Virtual Hard Disks & Checkpoint Management 7.6 Hyper-V Virtual Switches, NIC Teaming & Switch Embedded Teaming (SET) 7.7 Integration Services, Guest Components & Time Synchronization
9 Chapter 8: Hyper-V High Availability, Security & Containers
8.1 Hyper-V Failover Clustering & Cluster Shared Volumes (CSV) 8.2 Hyper-V Replica & Azure Site Recovery (ASR) 8.3 Guarded Fabric & Shielded Virtual Machines 8.4 Windows Server Container Hosts, Isolation Models & Images 8.5 Managing Container Instances & Linux Containers with WSL 8.6 Windows Container Networking & AKS Enabled by Azure Arc 8.7 Managing Container Images, Tags & Azure Container Registry
10 Chapter 9: Azure IaaS Virtual Machine Administration
9.1 Azure VM Sizing, Series Selection & Managed Disk Tiers 9.2 Azure VM Host Caching, Disk Bursting, Ephemeral OS Disks & Disk Operations 9.3 Azure VM High Availability: Availability Sets, Zones & Proximity Placement Groups 9.4 Virtual Machine Scale Sets, Autoscale & Azure Dedicated Hosts 9.5 Azure VM Networking, NSGs & Connectivity
11 Chapter 10: Hybrid DNS Infrastructure & Name Resolution
10.1 Active Directory-Integrated DNS Architecture, Dynamic Updates & Zone Types 10.2 DNS Records, Aging & Scavenging and the GlobalNames Zone 10.3 DNS Forwarding, Conditional Forwarding & Azure DNS Zone Integration 10.4 Hybrid Name Resolution with Azure DNS Private Resolver 10.5 Windows Server DNS Policies & DNSSEC
12 Chapter 11: IP Addressing, DHCP & IPAM Management
11.1 DHCP Server Role, the DORA Lifecycle & Active Directory Authorization 11.2 DHCP Scopes, Option Hierarchy, Reservations & Relay Agents 11.3 DHCP High Availability & Failover Configurations 11.4 IP Address Management (IPAM) Deployment & Auditing 11.5 Resolving IP Address Issues in Hybrid Environments
13 Chapter 12: Hybrid Network Connectivity, Remote Access & Private Access
12.1 Remote Access Role, VPN Protocols & Site-to-Site VPN to Azure 12.2 Network Policy and Access Services (NPAS), RADIUS & Policy Processing 12.3 Azure Network Adapter & Azure Extended Network 12.4 Web Application Proxy & Microsoft Entra Application Proxy 12.5 Microsoft Entra Private Access & Global Secure Access 12.6 Azure Relay & Hybrid Connections
14 Chapter 13: File Services, DFS, SMB & Azure File Sync
13.1 Windows Server File Share Access, NTFS Permissions & Access-Based Enumeration 13.2 File Server Resource Manager: Quotas, File Screens & Storage Reports 13.3 DFS Namespaces (DFS-N) & DFS Replication (DFS-R) 13.4 BranchCache: Distributed Cache and Hosted Cache Modes 13.5 Modern SMB Protocols, SMB Multichannel & SMB over QUIC 13.6 Azure File Shares, Tiers & Hybrid Permissions 13.7 Azure File Sync, Cloud Tiering & Monitoring 13.8 Migrating DFS Namespaces & DFS Replication to Azure File Sync
15 Chapter 14: Windows Server Storage Infrastructure & High Availability
14.1 Disk Partitioning, Volumes & Enterprise File Systems (NTFS vs ReFS) 14.2 Data Deduplication Architecture, Usage Types & Management 14.3 Storage Spaces & Storage Spaces Direct (S2D) 14.4 Storage Replica & Storage Quality of Service 14.5 iSCSI Target Server, IQNs, CHAP Authentication & Multipath I/O
14.2 Data Deduplication Architecture, Usage Types & Management
Key Takeaways Explain This Topic AI
Data Deduplication is a post-processing engine: files are written normally and only later sliced into variable 32 KB to 128 KB chunks, compressed, and stored once in the chunk store.
Optimized files become reparse points pointing into the System Volume Information chunk store, so the volume reports far more logical data than physical capacity.
The Default usage type targets general file shares with a 3-day minimum file age; Hyper-V tunes for VDI; Backup optimizes large sequential virtual backup files immediately.
Deleting data does not free space until the Garbage Collection job runs and unreferenced chunks are reclaimed, which is why free space appears unchanged right after a large delete.
Deduplication is supported on NTFS data volumes only, not on the boot or system volume and not on ReFS.
Last updated: August 2026
- Data Deduplication Architecture, Usage Types & Management
- 1. Data Deduplication Architecture & Chunk Store
Data Deduplication in Windows Server is a post-processing, variable-chunk storage optimization engine. It identifies redundant data blocks across files on a volume, stores a single unique instance of each chunk in a compressed repository, and replaces redundant file content with pointers. +-----------------------------------------------------------------------------------+
- | DATA DEDUPLICATION CHUNKING PIPELINE |
- | |
- | ORIGINAL FILES ON VOLUME (e.g., D:\Data) |
- | +--------------------------+ +--------------------------+ |
- | | File A: [Ch1][Ch2][Ch3] | | File B: [Ch2][Ch3][Ch4] | |
- | +--------------------------+ +--------------------------+ |
- | | | |
- | v v |
- | [DEDUPLICATION ENGINE] --> Variable Chunking (32 KB - 128 KB, Avg 80 KB) |
- | --> SHA-256 Hashing & Redundancy Analysis |
- | | |
- | v |
- | CHUNK STORE (Hidden in System Volume Information\Dedup\ChunkStore) |
- | +--------------------------------------------------------+ |
- | | [Unique Chunk 1] [Unique Chunk 2] [Unique Chunk 3] [4] | (Compressed) |
- | +--------------------------------------------------------+ |
- | ^ |
- | | |
- | REPARSE POINTS (Stub Files on D:\Data) | |
- | +--------------------------+ +--------------------------+ |
- | | File A: [Ptr1][Ptr2][Ptr3| | File B: [Ptr2][Ptr3][Ptr4| |
- | +--------------------------+ +--------------------------+ |
- +-----------------------------------------------------------------------------------+
- How Data Deduplication Operates
Post-Processing Engine: Unlike real-time deduplication that consumes CPU cycles during active write streams, Windows Server Deduplication runs as a scheduled background job, preventing write latency during peak production hours.
Variable-Sized Chunking: Uses Rabin fingerprinting algorithms to divide file streams into variable-sized chunks between 32 KB and 128 KB (averaging approximately 80 KB). Variable chunking ensures that inserting data in the middle of a file only changes adjacent chunks rather than invalidating the entire downstream block structure.
Hashing & Compression: Computes a cryptographically secure SHA-256 hash for each chunk. Unique chunks are compressed using the XPRESS algorithm and committed to the Chunk Store .
Reparse Points (Stub Files): The original file in the directory structure is stripped of its data payloads and converted into a Reparse Point . When an application requests the file, the deduplication file system filter driver ( dedup.sys ) transparently intercepts the I/O, fetches the appropriate chunks from the Chunk Store, decompresses them in memory, and returns the stream to the application without user awareness.
2. Usage Types & Job Types
When enabling Data Deduplication on a volume, an administrator must select the appropriate Usage Type to match the underlying workload pattern.
Usage Types Comparison
Usage Type Target Workload Default Minimum File Age Optimization Mechanics
Default General Purpose File Servers, user home folders, project shares 3 Days (Files modified < 3 days are skipped) Standard background optimization; skips open/locked files
Hyper-V Virtual Desktop Infrastructure (VDI) running open .vhdx files 3 Days (same age gate as Default) Optimizes in-use and partial files, so open VHD/VHDX files are still processed; leverages VDI image commonality
Backup Virtualized backup applications (Microsoft DPM, Veeam, Commvault) 0 Days — the only usage type that optimizes a file the moment it is written Priority optimization; optimizes in-use files but not partial files. Tuned for large sequential backup streams
Important
Hyper-V Usage Type Constraint: In Windows Server, the HyperV usage type is supported only for VDI (Virtual Desktop Infrastructure) deployments or virtualized backup servers. Deduplication is NOT supported for general Hyper-V compute hosts running production line-of-business server virtual machines (such as SQL Server or Active Directory Domain Controllers) due to random I/O overhead.
Deduplication Job Types
Data Deduplication executes four distinct maintenance jobs: +-----------------------------------------------------------------------------------+
- | DATA DEDUPLICATION JOB TYPES |
- | |
- | JOB TYPE PURPOSE & BEHAVIOR |
- | ------------------+-------------------------------------------------------------|
- | Optimization | Scans candidate files, chunks, hashes, compresses unique |
- | (Optimization) | data into the Chunk Store, and installs reparse stubs. |
- | ------------------+-------------------------------------------------------------|
- | Garbage Collection| Identifies orphaned chunks in the Chunk Store that are no |
- | (GarbageCollection| longer referenced by any reparse points (after file deletes)|
- | | and physically frees disk space. |
- | ------------------+-------------------------------------------------------------|
- | Integrity Scrubbing| Validates CRC/hash integrity of all chunks in the Chunk |
- | (Scrubbing) | Store. If corruption is found, repairs chunks using mirrors |
- | ------------------+-------------------------------------------------------------|
- | Unoptimization | Undoes deduplication; fully hydrates all files from the |
- | (Unoptimization) | Chunk Store back into standard raw files. |
- +-----------------------------------------------------------------------------------+
- 3. Managing Data Deduplication with PowerShell # 1. Install the Data Deduplication Role Feature
- Install-WindowsFeature -Name FS-Data-Deduplication -IncludeManagementTools
- # 2. Enable Deduplication on a General Purpose File Share (Volume D:) with 5-day age
- Enable-DedupVolume -Volume 'D:' -UsageType Default
- Set-DedupVolume -Volume 'D:' -MinimumFileAgeDays 5 -ExcludeExtension 'iso','zip','mp4'
- # 3. Enable Deduplication on a Dedicated Backup Target (Volume E:)
- Enable-DedupVolume -Volume 'E:' -UsageType Backup
- Set-DedupVolume -Volume 'E:' -MinimumFileAgeDays 0
- # 4. Manually trigger an on-demand Optimization job with high priority
- Start-DedupJob -Volume 'D:' -Type Optimization -Priority High -Memory 50
- # 5. Execute Garbage Collection to reclaim disk space after large file deletions
- Start-DedupJob -Volume 'D:' -Type GarbageCollection -Priority High
- # 6. Execute an Integrity Scrubbing job to verify chunk store integrity
- # Default run checks critical metadata only; add -Full to validate all data
- Start-DedupJob -Volume 'D:' -Type Scrubbing
- # 7. Query Deduplication Status and Savings
- Get-DedupStatus -Volume 'D:' | Format-List
- Get-DedupVolume -Volume 'D:' | Format-Table Volume, FreeSpace, UsedSpace, SavedSpace, OptimizedFilesCount