Deduplication Agent
Identifies and merges duplicate company and contact records across all pipeline data sources.
What it is
Duplicates are not just a data quality problem. They are a revenue problem. A rep working two records of the same company wastes capacity. A scoring model that sees the same company twice inflates signal density. This agent removes that problem using multi-signal entity matching across all sources.
How it works
The agent applies a three-pass matching strategy. First, exact domain matching identifies clear duplicates. Second, fuzzy name matching with Levenshtein distance and token overlap catches name variants. Third, LLM-assisted disambiguation resolves edge cases like subsidiaries, acquired companies, and regional entities sharing a parent domain. For each duplicate cluster, a master record is elected based on data completeness and duplicates are merged with a full provenance log.
How it fits into the pipeline
Runs after data normalization and before any enrichment or scoring step. Ensures the Firmographic Enricher processes unique entities, the intent scoring model does not double-count signals, and the CRM Update Agent does not create duplicate records.