Study guide
6.3 Match-Merge & Identity Resolution
DAMA Certified Data Management Professional CDMP
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & DMBoK Framework
1.1 CDMP Exam Details & Strategy 1.2 The DMBoK Framework 1.3 Data Management Organization & Roles
2 Chapter 2: Data Governance (Part A)
2.1 Data Governance Principles & Drivers 2.2 Operating Models & Structures 2.3 Policies, Standards, and Procedures
3 Chapter 2: Data Governance (Part B)
2.4 Data Stewardship 2.5 Governance Metrics & Implementation
4 Chapter 3: Data Modeling and Design (Part A)
Section 3.1: Conceptual, Logical, and Physical Models Section 3.2: ERD Notations & Relational Modeling Section 3.3: Normalization
5 Chapter 3: Data Modeling and Design (Part B)
3.4 Dimensional Modeling 3.5 Naming Standards & Modeling Best Practices
6 Chapter 4: Data Quality (Part A)
Section 4.1: Data Quality Dimensions Section 4.2: Data Quality Policy & Rules Section 4.3: DMAIC & DQ Improvement Lifecycle
7 Chapter 4: Data Quality (Part B)
4.4 Root-Cause Analysis & Remediation 4.5 DQ Metrics, Tools, and Reporting
8 Chapter 5: Metadata Management (Part A)
5.1 Business, Technical, and Operational Metadata 5.2 Metadata Lineage & Impact Analysis 5.3 Repository Architectures
9 Chapter 5: Metadata Management (Part B)
5.4 Standards and Semantics 5.5 Metadata Governance & Integration
10 Chapter 6: Master and Reference Data Management
6.1 Master vs Reference Data 6.2 MDM Repository Architectures 6.3 Match-Merge & Identity Resolution 6.4 Reference Data Governance & Standards
11 Chapter 7: Data Warehousing and Business Intelligence
7.1 DW Architectures: Inmon vs Kimball 7.2 Data Warehousing Storage & Load Patterns 7.3 BI Delivery & OLAP 7.4 Data Marts & Enterprise Data Warehouses
12 Chapter 8: Data Architecture
8.1 Enterprise Data Architecture & Zachman Framework 8.2 Data Flow Diagrams & Value Chains 8.3 Architecture Frameworks & Standards
13 Chapter 9: Document and Content Management
9.1 Records Management & ECM 9.2 Retention, Disposal, and E-Discovery 9.3 ARMA GARP Principles & Taxonomies
14 Chapter 10: Data Integration and Interoperability
Section 10.1: Integration Patterns & Technologies Section 10.2: Replication & Virtualization Section 10.3: Canonical Models & Integration Standards
15 Chapter 11: Data Security
Section 11.1: Security Principles: CIA Triad & Privacy Section 11.2: Access Control & Encryption Section 11.3: Security Regulations & Standards
16 Chapter 12: Data Storage and Operations
Section 12.1: Database Technologies & DBA Roles Section 12.2: Backup, Recovery, and High Availability Section 12.3: Performance Tuning & Operations
17 Chapter 13: Big Data and Data Science
13.1 Big Data V's & Technologies
18 Chapter 14: Data Ethics
14.1 DMBoK Data Ethics Framework
19 Chapter 15: Data Management Process and Maturity
15.1 Maturity Assessment Frameworks
6.3 Match-Merge & Identity Resolution
Key Takeaways Explain This Topic AI
Deterministic matching uses rule-based logic to require exact matches, prioritizing high precision but yielding low recall.
Probabilistic matching calculates statistical similarity scores using algorithms like Levenshtein Distance and Jaro-Winkler, maximizing recall.
Blocking keys (match keys) partition datasets to avoid the exponential performance bottleneck of O(N^2) pairwise comparisons.
Survivorship rules determine BVT creation by applying logic such as source system trust scores, recency, completeness, or frequency.
Last updated: July 2026
Match-Merge and Identity Resolution
At the heart of any Master Data Management ( MDM ) implementation is Identity Resolution . This is the process of identifying, matching, and merging records from disparate source systems that represent the same real-world entity (e.g., the same customer, product, or location). Without robust identity resolution, an organization cannot achieve a single, reliable Golden Record (or Best Version of Truth or BVT ).
The identity resolution engine operates through a structured process known as the Match-Merge Lifecycle . This lifecycle consists of five key phases: Data Standardization, Candidate Selection (Blocking), Matching, Survivorship, and Merging.
- The Match-Merge Lifecycle Phases
- 1. Data Standardization and Cleansing
Before records can be matched, they must be formatted consistently. Raw data is often plagued by typos, varying formats, and noise. Standardization involves:
Parsing : Splitting unstructured fields into structured attributes (e.g., splitting a full address string into Street Number, Street Name, Unit, City, State, and Zip Code).
Normalization : Converting data values to standard representations (e.g., converting "Avenue", "Ave.", and "AVE" to "Ave").
Casing and Punctuation : Removing special characters, trailing spaces, and standardizing case (e.g., " john.smith@domain.com " to " john.smith@domain.com ").
2. Candidate Selection and Blocking
Comparing every record in a database against every other record is computationally expensive. For a database with N records, a naive pairwise comparison requires N(N-1)/2 comparisons—an O(N^2) complexity. If a database contains 1,000,000 records, this would require nearly 500 billion comparisons.
To solve this, MDM systems use Blocking Keys (also known as match keys). Blocking partitions the database into smaller, manageable subsets (blocks) based on shared attributes (e.g., sharing the same first letter of the last name and the first three digits of the postal code). The matching algorithms are then run only within each block, drastically reducing the search space and execution time.
3. Matching Methodologies
Once candidates are grouped, the MDM engine applies matching rules to determine the likelihood that two records represent the same entity. There are two primary matching methodologies:
Deterministic Matching
Deterministic Matching relies on explicit, predefined rules and requires exact matches on specific data elements.
How it works : Rule-based logic (e.g., if "Social Security Number (SSN) matches" or "First Name + Last Name + Date of Birth matches exactly").
Pros : Extremely high precision (very few false positives), predictable, computationally fast.
Cons : Low recall (misses true matches due to minor spelling errors, nicknames, or missing fields).
Probabilistic Matching
Probabilistic Matching uses statistical models to calculate the mathematical probability that two records match. It assigns positive or negative weights to individual attributes based on their uniqueness and reliability.
String Similarity Algorithms :
Levenshtein Distance : Measures the minimum number of single-character edits (insertions, deletions, substitutions) required to change one string into another (e.g., "Smith" and "Smyth" has a Levenshtein distance of 1).
Jaro-Winkler : Measures similarity between two strings, giving higher weights to prefixes.
Phonetic Algorithms (Soundex / Double Metaphone) : Translates words into codes representing how they sound in English, allowing matches despite spelling differences (e.g., "Robert" and "Rupert").
Matching Thresholds :
Match Threshold : If the total similarity score exceeds this upper boundary, the records are matched and merged automatically.
Review Threshold : If the score falls between the match and non-match thresholds, the pair is routed to a Data Steward queue for manual review.
Non-Match Threshold : Below this boundary, the records are declared distinct.
4. Survivorship Rules
After identifying a match, the system must create the consolidated Golden Record. Survivorship Rules define how the MDM engine selects which attribute values "survive" from the source records to populate the golden record. Common survivorship strategies include:
Source System Trust : Fields from a highly trusted system (e.g., a customer portal where the customer typed their own email) take precedence over other systems (e.g., a legacy billing system).
Recency (Most Recent) : The attribute value with the latest update timestamp is selected.
Frequency (Most Common) : The value that appears most frequently across all matched source records is chosen.
Completeness : Choosing the value that is non-null, longest, or conforms to a validation mask.
5. Merging and Lineage
When records are merged, the MDM hub creates a new golden record or updates an existing one. Crucially, the hub must preserve the cross-reference (XREF) links to the original source records. This ensures Data Lineage is maintained. If matching rules are modified or a merge is discovered to be incorrect, the MDM system can execute an Unmerge operation, decomposing the golden record back into its constituent source records without data loss.
False Positives vs. False Negatives
In identity resolution, tuning the thresholds is a delicate balancing act:
False Positives : The system incorrectly matches and merges two records that actually represent different entities (e.g., merging the records of two different people named "John Smith" who live in different cities). This is a critical quality issue that can lead to privacy violations (disclosing one person's data to another).
False Negatives : The system fails to match two records that represent the same entity, leaving them as separate duplicates. This results in an incomplete view of the customer or asset.
Data management professionals must continually profile data and adjust matching weights to minimize both errors, prioritizing privacy and compliance.
DAMA Exam Traps and Study Tips
Precision vs. Recall : Understand that deterministic matching favors high precision (avoiding incorrect matches) at the expense of recall. Probabilistic matching increases recall (finding more matches) but increases the risk of false positives.
The Purpose of Blocking : The exam frequently tests why blocking is used. Remember: blocking is a performance-optimization technique used to reduce the number of record comparisons, preventing exponential performance degradation.
Role of the Data Steward : Understand that data stewards do not manually match every record. They only resolve the "gray area" exceptions that fall between the automatic Match and Non-Match thresholds.