Faster bitmaps

I've built an engine to bitmap data from a SQL server table for rapid analysis. Query times across 41 million rows with 58 attributes < 5 seconds on a Pentium II 233. Not bad!

However it takes forever to build the bitmaps. I've outlined (in pseudocode) the process I use to build the bitmaps below. The build on the above dataset takes 72 hours, anyone know of a quicker way?

For each atttribute (column)

    SELECT DISTINCT Values, Count(Values)     For each Value

        create empty bitmap
        SELECT URN Where Attribute =  Value
        Turn bit on for each URN returned above

I've got indexes on all of the high cardinality attributes, and take obvious shortcuts such as calculating the last bitmap by XORing all of the others. I need a fundamental algorithm change, not a tweak!

Thanks in advance
Dave Arkley Received on Mon Apr 16 2001 - 15:16:13 CEST