Home » SQL & PL/SQL » SQL & PL/SQL » Optimization ideas in view creation of subselect (10g R2 x64 , windows 2008 enterprise )
Optimization ideas in view creation of subselect [message #634864] Tue, 17 March 2015 03:22 Go to next message
clementstore
Messages: 11
Registered: December 2011
Junior Member
Hello guys,

I m joining 2 tables table_A and table_B into a view with a conditioned sub select (based on checktype) statement as below.
There are over 50 new columns that are generated by the sub-select statements in the view.

There is no choice to minimize the number of columns in the created View.
What would be the most efficient way to create this view ?

Thanks,
Clement


SELECT tab_A.id as ID, NVL (-tab_A.localcreditamount, 0) 
              AS balance,
           (SELECT checkvalue
              FROM table_B tab_B
             WHERE     checktype = '00010000000000000002'
                   AND tab_A.id = tab_B.id)
              AS pk_dept,
           (SELECT valuecode
              FROM table_B tab_B
             WHERE     checktype = '00010000000000000002'
                   AND tab_A.id = tab_B.id)
              AS dept_code,
           (SELECT checkvalue
              FROM table_B tab_B
             WHERE     checktype = '00010000000000000006'
                   AND tab_A.id = tab_B.id)
              AS pk_invcl,
           (SELECT valuecode
              FROM table_B tab_B
             WHERE     checktype = '00010000000000000006'
                   AND tab_A.id = tab_B.id)
              AS invcl_code
              .
              .
              .
              .
              .
FROM table_A tab_A 


--mod update: added [code] tags, please do so yoursef in future.

[Updated on: Tue, 17 March 2015 03:30] by Moderator

Report message to a moderator

Re: Optimization ideas in view creation of subselect [message #634865 is a reply to message #634864] Tue, 17 March 2015 03:27 Go to previous messageGo to next message
John Watson
Messages: 8931
Registered: January 2010
Location: Global Village
Senior Member
select id,checkvalue, ... from tab_a join table_b using (id) where checktype = '00010000000000000002';
Re: Optimization ideas in view creation of subselect [message #634867 is a reply to message #634864] Tue, 17 March 2015 03:35 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Never use SELECT subqueries in SELECT statement unless you are a SQL expert.

Re: Optimization ideas in view creation of subselect [message #634870 is a reply to message #634867] Tue, 17 March 2015 04:14 Go to previous messageGo to next message
clementstore
Messages: 11
Registered: December 2011
Junior Member
Thanks guys. Seems like I need a couple of joins to complete this as there are more than one checktype ( like 20 checktype value ) for the conditions?
Re: Optimization ideas in view creation of subselect [message #634871 is a reply to message #634870] Tue, 17 March 2015 04:29 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

We need a representative test case to answer you.

Re: Optimization ideas in view creation of subselect [message #634872 is a reply to message #634871] Tue, 17 March 2015 04:45 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
You need a join per checktype value.
Re: Optimization ideas in view creation of subselect [message #634902 is a reply to message #634872] Tue, 17 March 2015 17:46 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
if you do straight joins, you will need to use outer-join in case a checkvalue does not exist for some ID.

This appears to me to be a question of how you want to pivot the data. Consider what this might do for you:

      (
        select id
              ,min(case when checktype = '00010000000000000002' then checkvalue end) pk_dept
              ,min(case when checktype = '00010000000000000002' then checkvalue end) dept_code
              ,min(case when checktype = '00010000000000000006' then checkvalue end) pk_invcl
              ,min(case when checktype = '00010000000000000006' then checkvalue end) invcl_code
        ...
        from table_b
        group by id
      ) tab_b


Of course this presumes that the key of your table is ID,checktype. If not then you loose data. But then again if this is not the key of the demonstration table in your example then your example is wrong so...

I would expect this to be substantially faster if Oracle actually did the grouping first since it would result in much less index based I/O. Again I assume that your scalar sub-queries would be using an index to do a lookup and so you would have lots of logical I/O going on. Since there is little filtering happening in the example, lots of logical I/O seen inappropriate and so a full table scan of table_B along with its aggregation should be way faster than either the scalar sub-queries or the alternative straight up join strategy.

By the way, I believe Oracle's newer releases 11g/12c know how to translate scalar sub-queries into traditional joins so how you write it may not matter any more. I seem to recall seeing query plans that did a transform from scalar sub-query to simple join. Then again I may have eaten too many beans with my Mexican this weekend.

Kevin

[Updated on: Tue, 17 March 2015 17:58]

Report message to a moderator

Re: Optimization ideas in view creation of subselect [message #634912 is a reply to message #634902] Tue, 17 March 2015 22:30 Go to previous messageGo to next message
clementstore
Messages: 11
Registered: December 2011
Junior Member
Thanks all. This really seems to be very efficient. U all have been really helpful.
I just ran it and the result shows "[Error] Execution (3: 8): ORA-01652: unable to extend temp segment by 16 in tablespace"
which is at 32GB now. I will try to see if this can be fixed. Cheers!
Re: Optimization ideas in view creation of subselect [message #634913 is a reply to message #634912] Tue, 17 March 2015 22:37 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
[oracle@localhost ~]$ oerr ora 1652
01652, 00000, "unable to extend temp segment by %s in tablespace %s"
// *Cause:  Failed to allocate an extent of the required number of blocks for
//          a temporary segment in the tablespace indicated.
// *Action: Use ALTER TABLESPACE ADD DATAFILE statement to add one or more
//          files to the tablespace indicated.
Re: Optimization ideas in view creation of subselect [message #634943 is a reply to message #634913] Wed, 18 March 2015 04:47 Go to previous messageGo to next message
clementstore
Messages: 11
Registered: December 2011
Junior Member
Thank you very much for the inputs. I tried Kevin Meade 's approach and the outcome data was correct. Surprisingly , it didnt outperform the original subselect approach significantly, perhaps by 10% or so. I used 3 newly created temp files which amounts to 33GB on the first 500 rows.

Just on a brain storm, on a maybe similar occasion before, would the XML parsing method be useful in transforming row to column. I used this method before.
Just that I dont know how to apply to my case with the keys taking into consideration. Will try again...
Re: Optimization ideas in view creation of subselect [message #634944 is a reply to message #634943] Wed, 18 March 2015 04:54 Go to previous messageGo to next message
John Watson
Messages: 8931
Registered: January 2010
Location: Global Village
Senior Member
If you show the execution plan, preferably with execution statistics, perhaps someone can suggest how to tune the SQL.
Re: Optimization ideas in view creation of subselect [message #634962 is a reply to message #634944] Wed, 18 March 2015 07:19 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
Yes, read the article we have on what information is needed to tune a SQL Query and then post the BASIC information. We can have a look at the query plan from that. It will provide a good start.

Also, it sounds like you have lots of rows to group, or wide rows to group? You never told us how much data you have. Here is another thought which would be interesting to try out. We might all learn something. If the group by is the issue, then I wonder if we can use partitioning to reduce the size of the sort or hash operation that does it. For example, I would suggest the following:

If possible, create a version of the table that is HASH PARTITIONED (say into 128 partitions) using ID.  Since this will make the rows semi-grouped by ID (all rows for any single ID (eg. group we are creating) will exist in only one partition), Oracle should know that when grouping, each partition can be done independently of the other partitions.  Thus the group operation will require maybe 33GB/128 or 1/4GB of memory do group each partition.  If Oracle is smart enough, we won't need any disk space to do the group by.


Kevin
Good luck.
Re: Optimization ideas in view creation of subselect [message #635007 is a reply to message #634962] Wed, 18 March 2015 22:15 Go to previous message
clementstore
Messages: 11
Registered: December 2011
Junior Member
Yeah I have like ard 18 million rows to group into over 100 columns. After grouping, the row of this table will be mapped with another larger table on the ID field . These 2 tables should total no more 20GB inlcuding index. Thanks Kevin for your advice. I would have to work with the reality that our Oracle doesnt have partitioning with standard edition. I will also try to seek any other way for improvement.

Cheers
Clement
Previous Topic: row to column convert
Next Topic: Handle duplicates during direct load path in SQL LOADER
Goto Forum:
  


Current Time: Thu Apr 25 06:29:11 CDT 2024