Re: Re: Processing new rows in staging table quickly...

I agree. It is better not to overload the meaning of the date column. The date should be the date when the row was created and have an other columm for processed. Consider the request to reprocess the data of a specif day. You you would search for the day the data arrived, and not when it was processed.
----Ursprüngliche Nachricht----

Von : niall.litchfield_at_gmail.com
Datum : 18/04/2019 - 10:44 (GMT)
An : post.ethan_at_gmail.com
Cc : oracle-l_at_freelists.org
Betreff : Re: Processing new rows in staging table quickly...  The FBI approach will work just fine. Personally, I'd probably do one of the following:                

add a "PROCESSED" column accepting 'Y' or 'N' and index/query on that, Mostly for human readability.    

ask, if this is a "staging" table why are there ever any "processed" rows left in it.                 

  On Thu, Apr 18, 2019 at 12:33 AM <
  post.ethan_at_gmail.com> wrote:                   

I have a staging table in which 99% of the rows are going to have a date for stats_processed_time.            

I will need to only select for rows that have not been processed.            

What is the best strategy here? Should I allow nulls and select like below? Or should I put a date, way out in the future, say 2099, and select for that instead and then update to correct data once things are processed? Perhaps allow nulls and a bitmap or FBI solution (I would like to avoid solutions which can’t be ported to Postgres.)            

…     

where b.access_key=v_access_key     

       and b.bucket=v_bucket     

       and b.stats_processed_time is null           

--

 Niall Litchfield  

Oracle DBA  

 http://www.orawin.info

--

http://www.freelists.org/webpage/oracle-l Received on Thu Apr 18 2019 - 10:51:55 CEST