Optimization ideas in view creation of subselect [message #634864] |
Tue, 17 March 2015 03:22 |
|
clementstore
Messages: 11 Registered: December 2011
|
Junior Member |
|
|
Hello guys,
I m joining 2 tables table_A and table_B into a view with a conditioned sub select (based on checktype) statement as below.
There are over 50 new columns that are generated by the sub-select statements in the view.
There is no choice to minimize the number of columns in the created View.
What would be the most efficient way to create this view ?
Thanks,
Clement
SELECT tab_A.id as ID, NVL (-tab_A.localcreditamount, 0)
AS balance,
(SELECT checkvalue
FROM table_B tab_B
WHERE checktype = '00010000000000000002'
AND tab_A.id = tab_B.id)
AS pk_dept,
(SELECT valuecode
FROM table_B tab_B
WHERE checktype = '00010000000000000002'
AND tab_A.id = tab_B.id)
AS dept_code,
(SELECT checkvalue
FROM table_B tab_B
WHERE checktype = '00010000000000000006'
AND tab_A.id = tab_B.id)
AS pk_invcl,
(SELECT valuecode
FROM table_B tab_B
WHERE checktype = '00010000000000000006'
AND tab_A.id = tab_B.id)
AS invcl_code
.
.
.
.
.
FROM table_A tab_A
--mod update: added [code] tags, please do so yoursef in future.
[Updated on: Tue, 17 March 2015 03:30] by Moderator Report message to a moderator
|
|
|
|
|
|
|
|
Re: Optimization ideas in view creation of subselect [message #634902 is a reply to message #634872] |
Tue, 17 March 2015 17:46 |
|
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
if you do straight joins, you will need to use outer-join in case a checkvalue does not exist for some ID.
This appears to me to be a question of how you want to pivot the data. Consider what this might do for you:
(
select id
,min(case when checktype = '00010000000000000002' then checkvalue end) pk_dept
,min(case when checktype = '00010000000000000002' then checkvalue end) dept_code
,min(case when checktype = '00010000000000000006' then checkvalue end) pk_invcl
,min(case when checktype = '00010000000000000006' then checkvalue end) invcl_code
...
from table_b
group by id
) tab_b
Of course this presumes that the key of your table is ID,checktype. If not then you loose data. But then again if this is not the key of the demonstration table in your example then your example is wrong so...
I would expect this to be substantially faster if Oracle actually did the grouping first since it would result in much less index based I/O. Again I assume that your scalar sub-queries would be using an index to do a lookup and so you would have lots of logical I/O going on. Since there is little filtering happening in the example, lots of logical I/O seen inappropriate and so a full table scan of table_B along with its aggregation should be way faster than either the scalar sub-queries or the alternative straight up join strategy.
By the way, I believe Oracle's newer releases 11g/12c know how to translate scalar sub-queries into traditional joins so how you write it may not matter any more. I seem to recall seeing query plans that did a transform from scalar sub-query to simple join. Then again I may have eaten too many beans with my Mexican this weekend.
Kevin
[Updated on: Tue, 17 March 2015 17:58] Report message to a moderator
|
|
|
|
|
|
|
Re: Optimization ideas in view creation of subselect [message #634962 is a reply to message #634944] |
Wed, 18 March 2015 07:19 |
|
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
Yes, read the article we have on what information is needed to tune a SQL Query and then post the BASIC information. We can have a look at the query plan from that. It will provide a good start.
Also, it sounds like you have lots of rows to group, or wide rows to group? You never told us how much data you have. Here is another thought which would be interesting to try out. We might all learn something. If the group by is the issue, then I wonder if we can use partitioning to reduce the size of the sort or hash operation that does it. For example, I would suggest the following:
If possible, create a version of the table that is HASH PARTITIONED (say into 128 partitions) using ID. Since this will make the rows semi-grouped by ID (all rows for any single ID (eg. group we are creating) will exist in only one partition), Oracle should know that when grouping, each partition can be done independently of the other partitions. Thus the group operation will require maybe 33GB/128 or 1/4GB of memory do group each partition. If Oracle is smart enough, we won't need any disk space to do the group by.
Kevin
Good luck.
|
|
|
Re: Optimization ideas in view creation of subselect [message #635007 is a reply to message #634962] |
Wed, 18 March 2015 22:15 |
|
clementstore
Messages: 11 Registered: December 2011
|
Junior Member |
|
|
Yeah I have like ard 18 million rows to group into over 100 columns. After grouping, the row of this table will be mapped with another larger table on the ID field . These 2 tables should total no more 20GB inlcuding index. Thanks Kevin for your advice. I would have to work with the reality that our Oracle doesnt have partitioning with standard edition. I will also try to seek any other way for improvement.
Cheers
Clement
|
|
|