Re: Change Data Capture for Processes and not other databases
From: Dba DBA <oracledbaquestions_at_gmail.com>
Date: Tue, 12 Mar 2013 15:04:15 -0400
Message-ID: <CAE-dsO+MQs_j=JewZQyNrj8ipHYjA+m_ESt-k4+MzJQxqK6YOw_at_mail.gmail.com>
I don't like AQ for this. Here are my reasons. My understanding is that CDC is built on top of queues which is why I thought of it. 1. I can't guarantee the consumed records will make it to the downstream DB. We do not have a 3 phase commit. So I need to leave open the possibility to get new records
2. I have to store the records in a table anyway. So I have to insert the records and push them to a queue. Rate of insert is about 200 records/second. A downstream DB may come online and need to get a broad range of data.
Plus by rule we have to store it.
3. I have thousands of consumers. I am concerned about testing this since I don't have a way to simulate even a fraction of this. 4. Another company has the production support contract. The level of support we get for this is basically 'db is up so you are good'. anything else requires tickets, escalation requests, drama, and issues. I have supported
AQ in the past and occassionaly have had some issues with it. Usually developer created. Concerned about the operational support on this. 5. I have very strict purging rules. Not sure if I can properly enforce this with queues. That means if someone does not consume the data (have to purge every hour for data that is a few days old) then it has to go. I am purging with partitioning and my purge date stamp is not the same as when I get the data. I am concerned about mass popping data from queues. This is basically just an ETL database to temporarily house data. 6. We are into the development cycle and my developers would prefer to only have to adjust sql. if I have them pop, then they have to format the data. There are time issues.
Date: Tue, 12 Mar 2013 15:04:15 -0400
Message-ID: <CAE-dsO+MQs_j=JewZQyNrj8ipHYjA+m_ESt-k4+MzJQxqK6YOw_at_mail.gmail.com>
I don't like AQ for this. Here are my reasons. My understanding is that CDC is built on top of queues which is why I thought of it. 1. I can't guarantee the consumed records will make it to the downstream DB. We do not have a 3 phase commit. So I need to leave open the possibility to get new records
2. I have to store the records in a table anyway. So I have to insert the records and push them to a queue. Rate of insert is about 200 records/second. A downstream DB may come online and need to get a broad range of data.
Plus by rule we have to store it.
3. I have thousands of consumers. I am concerned about testing this since I don't have a way to simulate even a fraction of this. 4. Another company has the production support contract. The level of support we get for this is basically 'db is up so you are good'. anything else requires tickets, escalation requests, drama, and issues. I have supported
AQ in the past and occassionaly have had some issues with it. Usually developer created. Concerned about the operational support on this. 5. I have very strict purging rules. Not sure if I can properly enforce this with queues. That means if someone does not consume the data (have to purge every hour for data that is a few days old) then it has to go. I am purging with partitioning and my purge date stamp is not the same as when I get the data. I am concerned about mass popping data from queues. This is basically just an ETL database to temporarily house data. 6. We are into the development cycle and my developers would prefer to only have to adjust sql. if I have them pop, then they have to format the data. There are time issues.
On Tue, Mar 12, 2013 at 2:35 PM, rjamya <rjamya_at_gmail.com> wrote:
> wouldn't then AQ make perfect sense? You enqueue rows on commit and
> dequeue them upon reading. to me that would be a simpler solution.
>
> Raj
>
>
-- http://www.freelists.org/webpage/oracle-lReceived on Tue Mar 12 2013 - 20:04:15 CET