Re: A Question on Integrety

From: Tobes \(Breath\) <"Tobes>
Date: Wed, 17 Dec 2003 12:04:06 -0000
Message-ID: <brpgnq$5ph26$>

"--CELKO--" <> wrote in message
> I just sent in a three part article to on this topic. My
> recommendation was a series of "staging tables" which start off with
> the absolute nightmare of making all the columns VARCHAR(<max>) or
> NVARCHAR(<max>) and then move them in steps to other staging tables
> with more and more constraints and proper datatypes as I scrub the
> crap into some kind of usable shape.

Thanks for the reply, Joe.

I have been pondering this approach - and it may be the one I end up with. I'm a little reluctant since there might be several related tables that would all need similar treatment, which could give me an explosion of staging tables! Another consideration is that when applications are developed to talk to the database, they will need extra work to accommodate the various staging tables. I think there's no perfect option here, as usual it's just going to be a matter of trade-offs! I look forward to reading your article!

> Ideally, you'd like ETL tool that goes from raw files to the actual
> schema in one leap, but the intermediate "staging tables" let you
> "rollback" to the previous data quality level if you need to look at
> something -- a scrubbing audit trail.

The scrubbing audit trail could be useful in my circumstances, but I wasn't aware of ETL tools (or at least that category of tool)- so an interesting thing to consider


Tobin Received on Wed Dec 17 2003 - 13:04:06 CET

Original text of this message