Oracle-L: RE: Datawarehousing help

From: Kimberly Smith <ksmith2_at_myfirstlink.net>
Date: Fri, 03 May 2002 18:18:23 -0800
Message-ID: <F001.00457E8A.20020503181823@fatcity.com>

Excellent dude.

-----Original Message-----

WILLIAMS
Sent: Friday, May 03, 2002 5:48 PM
To: Multiple recipients of list ORACLE-L

Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means.

To learn Data Warehousing, I would encourage you to just do some "Googling" and find good tutorials. An excellent newslist is dwlist. Instructions:

For help with list commands, send a message to <mailto:dwlist-request_at_datawarehousing.com> with the word "help" in the body of the message.

The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for "Ralph Kimball". He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time.

The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA.
The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use.

OLTP databases have no concept of "time". A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year.

So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only.

For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views.

In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right.

Okay, I've rambled along here too long. Hope that gets you off on the right foot.

-----Original Message-----

[mailto:Rachel_Carmichael_at_Sonymusic.com] Sent: Friday, May 03, 2002 5:08 PM
To: Multiple recipients of list ORACLE-L

Okay, my background is OLTP, but we are looking at a data warehousing project
here....

any and all help appreciated! Specifically:

does anyone have any experience with a product called "SAS Datawarehousing Administrator" (or SAS)?
how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc.

help?

Rachel

--

Please see the official ORACLE-L FAQ: http://www.orafaq.com
--

Author:
INET: Rachel_Carmichael_at_Sonymusic.com

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists


--------------------------------------------------------------------

To REMOVE yourself from this mailing list, send an E-Mail message

to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
--

Please see the official ORACLE-L FAQ: http://www.orafaq.com
--

Author: DENNIS WILLIAMS
INET: DWILLIAMS_at_LIFETOUCH.COM

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists


--------------------------------------------------------------------

To REMOVE yourself from this mailing list, send an E-Mail message

--

Please see the official ORACLE-L FAQ: http://www.orafaq.com
--

Author: Kimberly Smith
INET: ksmith2_at_myfirstlink.net

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists


--------------------------------------------------------------------

To REMOVE yourself from this mailing list, send an E-Mail message