Re: What is the best way to do this?

From: Yechiel Adar <>
Date: Mon, 04 Apr 2011 09:02:54 +0300
Message-id: <>

Based on your answers I think I will do this:
  1. Create a new table that contain the customer number and the 5 attributes that you need for each customer. I mean, do a pivot on the data in data map and create one row for each customer containing all the 5 fields you need. You will need to recreate, or update, this table whenever the data map table changed or when you add or delete a customer. If you have, in data map, old customers that has no data in ABC then you can filter out their mapping in this step.
  2. Read the new table. ordered by customer id, and construct dynamic sql to get only the fields you need from ABC.

This way, you do not read the big table and you accessed the fact table only once for each customer and you save, big time, on network and database work to put all the fields in your buffer. This method also allow you to use bulk read for data map, saving more time.

You did not mentioned what partitioning you have, but if you partition data map according to the field meaning, I mean a partition for all "credit ratings", a partition for all "last order" etc,
You can replace the the table in step one with select the will bring you efficiently only the rows you need from the big table. If you have old data map for customers that has no rows in ABC, you can do: select * from data_map
where field_description in ('credit rating','last order') and customer_id in (select customer_pk from ABC);

Yechiel Adar

On 02/04/2011 02:03, Michael Moore wrote:
> Six questions:
> 1)How many customers?
> 26,000
> 2)Once established, can a customerís data_map change?
> it's very rare
> 3)If the answer to #2 is no, or only vary occasionally and with a
> defined maintenance event to reorganize that customerís data, how many
> different data_map rows do you have?
> 1,700,000
> 5)Do you have partitioning?
> yes

Received on Mon Apr 04 2011 - 01:02:54 CDT

Original text of this message