Re: 10 billion transactions a year?

On Sun, 22 Jul 2001, kusch_at_fuse.net wrote:

> We are looking at the need to build an application that can
> scale to upwards of 10 billion transactions/year.

Salivation :-)

> We've managed a data mart containing 2 billion transactions,
> but this new application will require a fair amount of complex
> processing to post the transactions

Are you expecting stored procs, java code, ...?

> (the data mart just reads/maps/plops) so isn't a good
> extrapolation of our knowledge.

I don't understand this one. You can't extend your datamart/warehouse to handle what you need, maybe adding additional hardware if needed?

Wouldn't this existing datastore have lots of data you would need to join to the new data?

> My two best guesses at an architecture to support something
> like this: a) a good sized computational cluster, essentially
> sharing the work

This sounds reasonable.

> b) some sort of master/slave organization with the slaves using
> database links to perform the majority of the processing, with
> the master being relegated to more or less a "posting machine"

DBLinks? Hm... I don't think these would be considered efficient. Large network traffic and joining to them will be a real bitch...

-- 
Galen Boyer
It seems to me, I remember every single thing I know.