Re: Datamart vs OLAP cubes?

From: joel garry <joel-garry_at_home.com>
Date: Tue, 30 Mar 2010 09:33:00 -0700 (PDT)
Message-ID: <049a95c2-d36e-4049-9f12-a3f164ea51b0_at_35g2000yqm.googlegroups.com>

On Mar 30, 12:55 am, guser78 <qazmlp1..._at_rediffmail.com> wrote:
> On Mar 30, 11:27 am, guser78 <qazmlp1..._at_rediffmail.com> wrote:
>
>
>
> > In some of the other posts that I have posted, I have mentioned the
> > business usecase that I'm dealing with.
> > Anyway, here is a brief overview of what we are working on:
>
> > Our Dataware house contains the data retrieved from many data
> > sources,  and all the data are somehow related to the information
> > specific to our Customers.
>
> > We have a marketing application running, and this requires summary
> > information on each of the Customer, like whether he is a ''Heavy
> > Buyer; Frequent buyer; Chocolate lover; Smoker'.
>
> > To calculate the above characteristics, we have to keep evaluating the
> > specific attributes in the Datawarehouse.
> > Also, due to the high performance requirements from the Marketing
> > application, we have planned to have a dedicated server to store the
> > pre-calculated characteristics information for all the subscribers.
> > This server should get periodically updated based on the latest
> > evaluation based on the data stored in the Data warehouse.
>
> > What do you suggest, for this dedicated server?
> >  - Maintaining a 'Data mart' of all the characteristics, for all the
> > Subscribers
> >  - Maintaining OLAP cubes, to represent all the  characteristics, for
> > all the Subscribers
>
> > Which one do you suggest among the above-mentioned approaches? Do you
> > have any other alternative suggestions? Why?
>
> > I humbly agree that I'm not a database architect. But, I promise to
> > go
> > into the details, once I get a clear picture on what way I should go
> > ahead with.
>
> Some of the experts might suggest to have both Datamart and OLAP
> cubes.
> But, my concern is that when the ETL layer which loads the data from a
> Data warehouse, can already to the pre-calculations and store the
> Customer characteristics directly in the Data mart. When this is
> already done, why should I require OLAP cubes additionally?
>
> Probably, now the comparison boils down to:
>  - ETL layer performing pre-calculations and storing into Data mart
>    VS
>  - ETL layer just extracting selected raw attributes into Datamart and
> forming OLAP cubes with the calculations
>
> Please pour-in your suggestions. Thanks

This can only be evaluated when you know what requirements require what when.

I've seen some situations that when stated out of context appear stupid, but aren't quite so stupid when most of the data is already moved to the cubes and only changes are propagating. So the initial load might take days, when Oracle could do it in less than an hour, but then again, it would take me months to replicate what people can just buy and run on SS, and the real problem is they want to be sold a solution without explaining specific requirements.

So the answer to Why? is often "because business requirements are fuzzy." Also, pre-calculating is an optimization, and you have to remember that optimizing too early is a mi$take. If you can kill two birds with one stone, fine, but be sure the birds are around. Personally, I've seen initial performance requirements for marketing analysis just be way out in left field, and always take a very critical eye to them (keeping the political realities in mind about when to say something). You have to have realistic windows for loading the intermediate data source, making the tertiary data source more restrictive than that can be silly.

jg

--
_at_home.com is bogus.
"Metrolink Heralds Arrival of Pilot Crash Energy Management-Enabled
Passenger Cars" - headline at metrolink.com

Received on Tue Mar 30 2010 - 11:33:00 CDT