Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> comp.databases.theory -> Faster bitmaps

Faster bitmaps

From: Dave Arkley <DaveArkley_at_REMOVE.THIS.lineone.net>
Date: Mon, 16 Apr 2001 14:16:13 +0100
Message-ID: <MeCC6.528$lL5.2086@NewsReader>

I've built an engine to bitmap data from a SQL server table for rapid analysis. Query times across 41 million rows with 58 attributes < 5 seconds on a Pentium II 233. Not bad!

However it takes forever to build the bitmaps. I've outlined (in pseudocode) the process I use to build the bitmaps below. The build on the above dataset takes 72 hours, anyone know of a quicker way?

For each atttribute (column)

    SELECT DISTINCT Values, Count(Values)     For each Value

        create empty bitmap
        SELECT URN Where Attribute =  Value
        Turn bit on for each URN returned above
    next value
next attribute

I've got indexes on all of the high cardinality attributes, and take obvious shortcuts such as calculating the last bitmap by XORing all of the others. I need a fundamental algorithm change, not a tweak!

Thanks in advance
Dave Arkley Received on Mon Apr 16 2001 - 08:16:13 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US