Destination marketers see each organisation and its meeting history once, not scattered across duplicate records.
The client wanted the destination marketing organisations it serves to find leads for future events in a database they could trust, without duplicates.
What changed
What stood in the way.
- The database held over 150,000+ meeting histories and 160,000+ organisations, many of them duplicates in short forms, numbers and Unicode.
- Finding duplicates by hand would take 60+ days, and a rule-based NodeJS approach would not keep its accuracy as data grew.
- 98.8%accuracy (as reported)
- 80%less time than finding and merging duplicates by hand (as reported)
- 300,000+records analysed (as reported)
What we built.
- Text analysis and cleaning of organisation names, then a BERT model that finds and merges similar organisations in 4-6 hours with 99% accuracy.
- A second model for meetings, chosen after testing CNN, sequential, random forest and decision tree models on 30,000 records.
- A side-by-side comparison with a NodeJS rule-based version to show the difference in accuracy.
- BERT
- NLP
- CNN
- Random forest
- Decision trees
- Node.js
What this work shows about us.
Several models were compared on real records, and the AI approach was shown against a rule-based one before it was chosen.
For a lead database, duplicates are the main cost to users, so the work went straight at them.
Want AI that finds and merges duplicate records in your data? Tell us what you want to change. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with the right team.
Start your plan ↗If we built it today, with AI.
What we would add now, moment by moment.
An agent would check each new organisation or meeting as it is added, suggest a likely match and ask a data steward to confirm the merge.
An agent would answer a destination marketer's question about which organisations have held events like theirs, from the cleaned history, and draft the outreach list for them to review.
Where this work fits.
Facts and figures from the project records; nothing added.
Want this for your business?
Tell us what you run or what you're building. Our AI assistant, Ask Unthinkable, helps you define your requirements, then connects you with a senior AI engineer.
Send us a note instead
An AI agent. This prototype uses prepared answers and sends nothing.
We use essential cookies to run this site. With your consent we also use analytics cookies to see which pages help. Privacy policy