Re: efficient compare

Andersen wrote:
> I have two sets A and B, each containing lots of key/value pairs. I want
> to compare them, to find out what is missing so that I can transfer
> whatever is needed from A to B and whatever is needed from B to A such
> that A=B. Any effective algorithms for that? Where should I look?
>
> Rephrasing it in math terminology:
> If I have two sets A, and B, containing tuples, and I want to find
> complement(A intersect B), how do I do that efficiently?

 From reading the clarifications made in subsequent posts to this discussion, it seems that you don't have two sets, you have one, and the issue isn't how to reconcile more than one set but how to maintain one set in more than one place.

Which begs the question: does it have to be more than one place? is there no acceptable way to connect data sources and sinks to one database?

Certainly, there are many good reasons to replicate data stores. If one of those is applicable and the answer is, "Yes, the data have to be in more than one place," then consistency and latency (among others) become significant potential issues.

It seems you've already answered this question and are trying to figure out how to maintain a distributed database. I suggest you check out the distributed databases already on the market (dismal performers) or check out Stonebraker's work-in-process (tolerates inconsistencies "for a while.") Received on Sun Apr 23 2006 - 19:03:20 CEST