DB Link or Temp Table? [message #353384] |
Mon, 13 October 2008 12:51  |
ehegagoka
Messages: 493 Registered: July 2005
|
Senior Member |
|
|
Not sure if this is a *bad* question. But just curious, if I were to fetch some data from another database (some big amount of data). Is it good to use dblink or is it much better to create staging table from the other database, populate it, and then just select from that staging table? Thanks.
|
|
|
|
|
Re: DB Link or Temp Table? [message #353477 is a reply to message #353384] |
Tue, 14 October 2008 00:45   |
 |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
if you are creating a staging table locally then there are some obvious points of interest:
Quote: | 1) both solutions require you to move the same amout of data across your network
2) both solutions require you to consume memory/disk space to house rows for use locally
|
so far not much difference
Quote: | 3) staging tables can be commited thus making the data viewable after use (good for debugging)
4) staging tables can be queried over and over without incurring the network overhead that would be required in referencing the data more than once on the db_link
5) staging tables can be indexed thus making access to them potentially faster if the rowset is indeed queried different ways
|
sounds good for staging tables so far, but...
Quote: | 6) staging tables require maintenance even if only to remember to truncate them when you are done with them, otherwise they continue to consume space with data not being used
7) staging tables are by definition a form a snapshot and thus are immediately stale after they are populated. This can be good if you don't want the picture to change, bad if you need to keep in synch with updates while you work with the data copy
|
So, the question is not should I use a link or staging table, but rather, how do the behavioral differences between the two strategies align to the data processing goals supporting my business need.
As for "slowing down the servers", what do they expect will be different when a staging table is used? If the data is referenced only once, then there will be little difference. Ask the people who told you "it slows down the servers" to explain their reasoning because I am not sure they know what they are talking about.
Good luck, Kevin
[Updated on: Tue, 14 October 2008 00:48] Report message to a moderator
|
|
|
|
|
Re: DB Link or Temp Table? [message #353639 is a reply to message #353477] |
Tue, 14 October 2008 07:48   |
ehegagoka
Messages: 493 Registered: July 2005
|
Senior Member |
|
|
Hi, Thanks for all the replies. The query on the "source db" runs for about 2-3 hours. It is a big summary query, which just runs once in a while for a sample day.
I've read an article yesterday saying also that the cons of doing the query over a database link would be the "processing of the data" (i.e. sorting, joins, etc.). Compared to the just pulling the process data from a staging table.
What I did now is create a staging table from the source db, then truncate that table every time I need to reload. Then just insert-select to our table on our database. I've put an index on that staging table. One thing I noticed is that when I ran the query from the source db, it is using the index (table has been analyzed). But when I ran that query from our database, adding the dblink, it just do a full table scan. Is that the expected behavior? Or do I need to configure something else?
Also not sure if this has already been answered. Is the running a query over a database link and running it directly on that source db have that much of a difference of putting the "pressure" on the source db? (not considering the load on network issue)
Thank you very much again.
[Updated on: Tue, 14 October 2008 07:52] Report message to a moderator
|
|
|
|
Re: DB Link or Temp Table? [message #353699 is a reply to message #353668] |
Tue, 14 October 2008 13:30   |
ehegagoka
Messages: 493 Registered: July 2005
|
Senior Member |
|
|
Hi, Thanks again. Here's how the current setup goes:
database A
create table..a_stg
procedure procA is
begin
execute immediate..truncate a_stg;
insert into a_stg
select sum()...etc.;
end;
We ran first this procA here in database A
Then in...
database B
procedure procB is
begin
insert into b_table
select ... from a_stg@databaseA;
end;
Run procB.
Thanks again.
|
|
|
|
Re: DB Link or Temp Table? [message #353707 is a reply to message #353702] |
Tue, 14 October 2008 15:10   |
ehegagoka
Messages: 493 Registered: July 2005
|
Senior Member |
|
|
Hi, Thanks again. I learned something new =)
I tried your cases, if this is the correct flow. Based on this, DDL operations are not allowed remotely if the DDL is being executed from the local site. Unless the DDL is executed ON the remote site.
Session 1:
create table t_at_db1(x int);
insert into t_at_db1 values(2);
Session 2:
create table t_at_db2(x int);
insert into t_at_db2 values(5);
create procedure p is
begin
execute immediate 'truncate table t_at_db2';
end;
/
Session 1:
SQL> begin
2 update t_at_db1 set x = 4;
3 execute immediate 'truncate table t_at_db2@link';
4 --p@link;
5 end;
6 /
begin
*
ERROR at line 1:
ORA-02021: DDL operations are not allowed on a remote database
ORA-06512: at line 3
Using procedure
SQL> begin
2 update t_at_db1 set x = 4;
3 --execute immediate 'truncate table t_at_db2@link';
4 p@link;
5 end;
6 /
PL/SQL procedure successfully completed.
|
|
|
Re: DB Link or Temp Table? [message #353709 is a reply to message #353707] |
Tue, 14 October 2008 16:01  |
 |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
good job. However I wonder if there is more (though maybe not for you).
I recall the commits on the remote site, during a distributed transaction where disallowed.
Maybe I am mixed up though.
Anyone else know?
Kevin
|
|
|