Re: Deduplication of records

Hi

I have been involved with this issue a number of times during data migrations i have performed for various clients over a number of years. I have used hand written C to do this as part of a C based data migration code, and i have used a commercial program. I think it was a component of quick address. This was quite a good tool that accepted comma seperated lists of names and addresses and did do quite a good job of identifying duplicates based on a number of rules based on parts of addresses and names and post codes.

The big thing that was evident each time I have been involved in de duping names and addresses is the issue of who is responsible for merging records or not based on automatic de-duping. remember this could result in wrong customers being billed, customers not being billed at all. In each case i have been involved in the code / tools used ended up being used to only highlight cases of duplication. Business users and temps hired did the de-duping by hand using the guidance of automatic tools.

HTH regards

Pete Finnigan
www.pentest-limited.com

In article <Xns911BB04B96FArdldssnl_at_213.222.27.9>, Raymond de Ligt <rdl_at_dss.nl> writes
>Hi Everybodu, does anyone have some information about how to deduplicate
>records in databases, example how does one prevent that he gets the same
>person Mr. Johnson more than once in the system , once as Mr. Johnson and
>the other time as Mr. Johnsson.
>
>I'm doing this on a Magic system and a oracle Dbase

-- 
Pete Finnigan
IT Security Consultant
PenTest Limited

Office  01565 830 990
Fax     01565 830 889
Mobile  07974 087 885

pete.finnigan_at_pentest-limited.com

www.pentest-limited.com