Study guide

1.4 Duplicate CI Handling and Remediation

ServiceNow Certified Implementation Specialist - Data Foundations CMDB and CSDM

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: CMDB Governance

1.1 CMDB Health Dashboard and the Three Cs 1.2 CMDB Data Manager and Lifecycle Policies 1.3 CSDM Lifecycle Stage Governance 1.4 Duplicate CI Handling and Remediation 1.5 Principal Classes, CMDB Workspace, and Operationalizing Governance

2 Chapter 2: CMDB Configuration & CI Class Manager

2.1 CI Class Manager and Hierarchy Architecture 2.2 Authoring Identification Rules

3 Chapter 3: CI Ingestion & the Identification and Reconciliation Engine

Section 3.1: Ingestion Methods and Service Graph Connectors Section 3.2: Identification and Reconciliation Engine (IRE) Core Section 3.3: Multisource CMDB (CMDB 360) Section 3.4: Non-Discoverable CIs, Manual Data, and Compliance Identifiers

4 Chapter 4: CMDB Insight & Querying

4.1 CMDB Query Builder 4.2 Dependency Views and the Unified Map 4.3 Relationship-Driven Impact Analysis 4.4 Natural Language Query and the CMDB Data Foundations Dashboard

5 Chapter 5: Common Service Data Model (CSDM)

Section 5.1: CSDM Framework and Foundation Domain Section 5.2: CSDM Design, Build, and Manage Technical Services Domains Section 5.3: Service Modeling and CI Alignment

1.4 Duplicate CI Handling and Remediation

Key Takeaways Explain This Topic AI

The Identification and Reconciliation Engine (IRE) automatically detects duplicate CIs during data ingestion and generates de-duplication tasks.

De-duplication tasks are stored in the reconciliation_duplicate_task table and group duplicate CIs around a selected Master CI.

The Duplicate CI Remediation wizard provides three resolution actions: Attribute/Relationship Merge, Direct Deletion, or Mark as Not Duplicate.

Marking records as 'Not Duplicate' writes to the cmdb_duplicate_ci_exclusion table to prevent the IRE from flagging them again.

Merging CIs consolidates history, tasks (like open incidents or changes), and relationships before retiring or deleting the duplicate records.

Last updated: July 2026

Duplicate CI Handling and Remediation

Data integrity is the cornerstone of a functional Configuration Management Database (CMDB). A common issue in large enterprises is the creation of duplicate configuration items (CIs)—multiple records in the CMDB representing a single physical asset or logical service. Duplicates typically occur due to misconfigured identification rules, multiple data sources (such as Discovery, Service Graph Connectors, and manual imports) writing to the CMDB without passing through the Identification and Reconciliation Engine (IRE), or manual data entry errors.

The consequences of duplicate CIs are severe. They lead to fragmented ticket history (incidents and changes attached to different records for the same server), broken service maps, inaccurate reporting, and failed software license compliance audits. Managing and remediating duplicate CIs is therefore a critical component of CMDB governance.

1. Detection via the Identification and Reconciliation Engine (IRE)

ServiceNow prevents and detects duplicate CIs primarily through the Identification and Reconciliation Engine (IRE) . The IRE uses class-specific Identification Rules (which contain Identifier Entries ) to evaluate incoming payloads. An identifier entry defines the unique attributes of a class, such as serial number, MAC address, hostname, or IP address.

When a discovery source attempts to insert or update a CI, the IRE evaluates the payload:

If the payload match criteria point to an existing record, the IRE updates that record.

If the payload matches multiple records with different system IDs ( sys_id ), the IRE identifies a duplication conflict.

The IRE allows the update to proceed on one record, but it automatically generates a De-duplication Task in the reconciliation_duplicate_task table to flag the conflict for administrator review.

De-duplication tasks are also generated by the CMDB Health Correctness Dashboard scheduled job, which scans the CMDB periodically to identify existing duplicates that may have bypassed the IRE (e.g., through direct database writes or legacy import sets).

2. The Duplicate CI Remediation Wizard

ServiceNow provides a guided, wizard-driven interface called the Duplicate CI Remediation tool to resolve de-duplication tasks. This tool ensures that duplicates are resolved safely, preserving transactional history and downstream relationships.

When an administrator opens a de-duplication task, the remediation wizard guides them through the following critical steps:

Identify the Master CI: The administrator must select one CI from the duplicate group to serve as the single source of truth (the Master CI). ServiceNow provides recommendations based on criteria such as the record with the oldest creation date, the record discovered by the most authoritative source, or the record with the highest density of attributes and active relationships.

Consolidate Attributes (Merge): The wizard allows the user to compare the attributes of the duplicate CIs side-by-side. If a duplicate record contains a more accurate or recent value for an attribute (e.g., a newer firmware version or updated location), the user can choose to merge that specific value into the Master CI.

Consolidate Related Items and Relationships: This is the most critical step of the remediation process. The wizard automatically moves all downstream and upstream relationships, as well as operational records (such as active Incidents, Problems, Change Requests, and Outages), from the duplicate CIs to the Master CI. This prevents the loss of historical data and ensures that active support tickets remain linked to the correct configuration item.

Decommission Duplicate CIs: Once attributes and relationships are consolidated, the wizard prompts the user to dispose of the redundant duplicate records. The duplicates can either be permanently deleted from the database or retired (updated to lifecycle_stage = End of Life and lifecycle_stage_status = Retired) according to the organization's data retention policies.

3. Alternative Remediation Paths

While merging is the standard resolution path, the Duplicate CI Remediation tool supports two alternative options depending on the nature of the duplicates:

Direct Deletion: If the duplicate records are verified to be empty, stale, or "junk" records with no active relationships or history, the administrator can bypass the merge process and delete them directly to clean up the database.

Mark as Not Duplicate (Exclusion): In some scenarios, the IRE may flag CIs as duplicates when they are actually distinct physical devices. This occurs if identification rules are too loose or if multiple devices share non-unique identifiers (e.g., virtual machines cloned with identical MAC addresses or serial numbers). If the administrator verifies that the CIs are indeed distinct, they can mark them as "Not Duplicate".

Marking records as "Not Duplicate" creates a record in the Duplicate CI Exclusion ( cmdb_duplicate_ci_exclusion ) table. The IRE references this table during subsequent ingestion cycles and bypasses duplicate checks for the specified CI pair, preventing them from being flagged in future de-duplication tasks.

Duplicate Resolution Paths Comparison

Resolution Path Target Table Impact on Redundant CIs Typical Scenario

Merge reconciliation_duplicate_task Attributes merged; duplicates retired or deleted Active duplicates with shared history and relationships

Direct Delete reconciliation_duplicate_task Redundant CIs purged immediately "Junk" records with no transactional history

Exclusion cmdb_duplicate_ci_exclusion CIs retained; marked as unique to bypass future checks False positives caused by shared MAC or serial numbers

By integrating the Duplicate CI Remediation wizard into governance workflows, CMDB administrators can systematically resolve data conflicts, maintain clean relationship maps, and ensure the overall accuracy of platform operations.

Loading diagram... Duplicate CI Remediation Process Flow