© omikron 2004 - © 1993-2006 omikron data quality gmbh ∙ ∙ [email protected] duplicates &...
TRANSCRIPT
© OMIKRON 2004 - www.omikron.net© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Duplicates & matching in worldwide data: Challenges &
Solutions
Carsten Kraus
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
The world becomes more global
Germany creates over 30% of its GNP by exports
In 1993, export accounted for only 20%
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
The world becomes more local
When conquered by the russians, Mongolia changed from their own alphabet to Cyrillic
In 1993, they changed back Ireland and many other countries put
efforts to strengthen their own languages Don‘t believe the world will soon switch to
English standards anyway
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
The World – so what?
Worldwide customer data needs:- A worldwide adequate data structure- worldwide adequate entry forms
e.g. for the internet
- Worldwide adequate Processing E.g. Matching / Duplicate check
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Why is matching important
Save money- Avoid duplicates in Mailings- Avoid selling to Blacklisted customers or above credit limit
Earn more money- Avoid customer frustation- Single view of customer needed for BI
CLV cross selling marketing controlling …
Save your life- Avoid selling to terrorists
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Do not trust the postal code
In the UK, the postal code goes very deep, sometimes to the level of a single building- It is therefore a strong identifier in duplicate checks
Germany, France, Switzerland, Sweden … only 5 resp. 4 numeric digits
70 countries do not have any postal code at all – e.g. Ireland
Do not trust that you can base strong identification on the postal code
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Postal Data
Street names are available only for a few countries
Different periods of updating Therefore: Duplicate Check must also
be able to handle addresses, which are not postally precise
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Characters in the world:
Alphabet
Abugidas
Abjads
Syllables Script
Symbol Script
- Александр Пушкин
- े�वना�गर
- الدن بن أسامة
- あいこ
- 愛子
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Processese.g. Duplicate Check
Just „Unicode capabilities“ are not enough
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Japan
Multiple ways of writing: Aiko (Child of Love) 愛子 あいこ
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
China ZHANG Aiguo 张爱国 ZHANG Aimin 张爱民 ZHANG Aidang 张爱党
ZHANG is the family name, Ai the generation name
Only the last syllable represents the given name
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
The 5 most common names
U.K. 3,3% China 31%
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Thus
Do not trust the family name to be a good differentiator in all countries
Your software should be able to handle these cases
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Householding
In many countries, male and female names have different endings- E.g. greek names
Male: Πέτρος Κώτης (Petros Kotis) Female Αναστασία Κώτη (Anastasia Koti)
When identifying households, it is just not enough to search for a 100%match
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Order of given/family name
In the U.K., names begin with the given name:- John Smith
In France and in many other countries the given name stands after the familiy name:- DUPONT Michel
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Komposita in Firmennamen
English: ring tone service Ltd. (3 words) German: klingeltonservice GmbH (1 word) Or:
- Klingelton-Service- Service für Klingeltöne
Not only German:- Netherlands- Scandinavia- Occasional occurrence in many languages
Most algorithms cannot solve that as they compare wordwise
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Omikron offers: Worldwide matching technology
- At D&B Sweden and at Schober Iberia, we replaced localized solutions because our international matching technology proved better results than localized solutions – even on local data
- At Reed, we found 20 000 more duplicates in 400 000 international Addresses already processed by another high end software
- Patent pending Other DQ technology
- e.g. data structuring, Upper/lower case… All built into an SOA-ready solution, the Omikron
DQ Server- Other surroundings available
© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Thus:
Handling international data correctly, means more than just being able to import Unicode-Data
Keep in mind the impact on- Data storage- Data entries- Matching- Salutation- etc.
The global world becomes more local again – care about it and you will have a competitive advantage
Feel free to ask us to help you ;-)
© OMIKRON 2004 - www.omikron.net© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Attention, content has been added/changed
For updated Presentation, please give me your card
© OMIKRON 2004 - www.omikron.net© 1993-2006 OMIKRON Data Quality GmbH ∙ www.global.omikron.net ∙ [email protected]
Thank you for your attention!
Carsten [email protected]