Re: Selecting SIMILAR, not the same records (PROBABLE) duplicates

From: Malcolm Dew-Jones <yf110_at_vtn1.victoria.tc.ca>
Date: 15 Sep 2006 18:47:52 -0700
Message-ID: <450b57c8$1@news.victoria.tc.ca>

kroger (kroger_at_vp.pl) wrote:
: Hi,

: I've been struggling with that for two days now...
: There is a simple solution for finding duplicates

I thought I had posted the following, but I think I mailed it by mistake since it bounced. (And apologies if this turns up multiple times as something odd seems to be up with our news reader setups).

The program you need is called AUTOMATCH, however I don't know if it is available anymore, or where you get it from. Google has references to it, so you can start from there.

AUTOMATCH uses statistical tests to group together similar data, and is based on sound mathematical principals, not just a bunch of ad-hoc tests . The version I saw did not work directly in the database, we extracted data, ran the compares and resolving steps, and then used the result to update the database.

Discussions of the mathematical theory behind it used to be available online, so even if the program isn't available anymore, google might help you to create a similar program for your situation. Received on Fri Sep 15 2006 - 20:47:52 CDT