Home » SQL & PL/SQL » SQL & PL/SQL » Performance Issue [Merged] (Oracle 11g Database, Win Xp)
Performance Issue [Merged] [message #623426] Wed, 10 September 2014 05:20 Go to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Hi,

I have performance issue in a Program,
actually the program has 2 delete statements
and one insert command.

well i tried to analyze that queries, i find that
in a delete statement they are using(in the Project)
substring 3 times in where condition, i like to know
how to avoid substring, instead what else we can use of it.

example query:

 select * from emp where
          substr(hiredate,1,7) < to_char(add_months(to_date(substr('2014-08-18',1,7),'YYYY-MM'),-16), 'YYYY-MM')
          and substr(hiredate,6,2) = '12'


any idea?
Re: Performance Issue [message #623431 is a reply to message #623426] Wed, 10 September 2014 05:35 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
what datatype is hiredate?
Re: Performance Issue [message #623432 is a reply to message #623426] Wed, 10 September 2014 05:36 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

    trunc(hiredate,'month')) < trunc(add_months(date '2014-08-18',-16),'month')
and extract(month from hiredate) = 12

[Updated on: Wed, 10 September 2014 05:37]

Report message to a moderator

Re: Performance Issue [message #623433 is a reply to message #623426] Wed, 10 September 2014 05:37 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
This is because of your poor design.

A DATE column should always be DATE DATA TYPE. Since you have HIREDATE column as character type, you are/and will always face issues. The first action item for you is to change the data type to DATE as soon as possible.
Re: Performance Issue [message #623435 is a reply to message #623431] Wed, 10 September 2014 05:38 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Its Varchar
Re: Performance Issue [message #623436 is a reply to message #623435] Wed, 10 September 2014 05:39 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

stalin4d wrote on Wed, 10 September 2014 12:38
Its Varchar


Aaaargh!

Re: Performance Issue [message #623440 is a reply to message #623432] Wed, 10 September 2014 05:42 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
Assuming hiredate is a date that's obviously better to the original but I doubt it'll make any difference to the performance problem.
Re: Performance Issue [message #623442 is a reply to message #623435] Wed, 10 September 2014 05:44 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
stalin4d wrote on Wed, 10 September 2014 11:38
Its Varchar


Then you're stuck with substr, poor performance and high probability of bugs.
Dates should always be stored in date columns.
Re: Performance Issue [message #623443 is a reply to message #623432] Wed, 10 September 2014 05:50 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Michel Cadot wrote on Wed, 10 September 2014 17:36

    trunc(hiredate,'month')) < trunc(add_months(date '2014-08-18',-16),'month')
and extract(month from hiredate) = 12



Im getting an error when applying this above code;

ORA-3007 Invalid extract field for extract source, 
Cause: The extract source does not contain the specified Extract field
Re: Performance Issue [message #623447 is a reply to message #623442] Wed, 10 September 2014 05:59 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
cookiemonster wrote on Wed, 10 September 2014 17:44
stalin4d wrote on Wed, 10 September 2014 11:38
Its Varchar


Then you're stuck with substr, poor performance and high probability of bugs.
Dates should always be stored in date columns.


What to do then, actually the customer/client needs to store the date for some specific reason,
in some specific format for their reports,

They by default store in this below format;

HireDate
----------
2007-12-31
2007-12-30

Re: Performance Issue [message #623448 is a reply to message #623443] Wed, 10 September 2014 06:01 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
stalin4d wrote on Wed, 10 September 2014 16:20

Im getting an error when applying this above code;

ORA-3007 Invalid extract field for extract source, 
Cause: The extract source does not contain the specified Extract field


SQL> select extract(month from sysdate) from dual;

EXTRACT(MONTHFROMSYSDATE)
-------------------------
                        9


Now, can you tel the reason why your code throws an error but my code works?
Re: Performance Issue [message #623449 is a reply to message #623447] Wed, 10 September 2014 06:02 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
stalin4d wrote on Wed, 10 September 2014 16:29

What to do then, actually the customer/client needs to store the date for some specific reason,
in some specific format for their reports


What is that specific reason and specific format?
Re: Performance Issue [message #623455 is a reply to message #623449] Wed, 10 September 2014 06:14 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
If the date needs to be in a specific format for a certain purpose then it should be formatted when selecting it for that purpose. That is not a good reason to store it as a varchar.

Michel's code doesn't work for you since he assumed the column was a date.

You could add function based index on substr(hiredate,1,7) and substr(hiredate,6,2), but really you should make the column the correct datatype.

[Updated on: Wed, 10 September 2014 06:14]

Report message to a moderator

Re: Performance Issue [message #623461 is a reply to message #623449] Wed, 10 September 2014 06:56 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Lalit i dont know the reason, but for spliting or taking parameter for reports
they might be taking store in that format.

i will try your extract month and tell...
Re: Performance Issue [message #623462 is a reply to message #623461] Wed, 10 September 2014 07:00 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
Don't bother.
Extract is function used on dates (as is trunc). You don't have a date. So there's no point you trying to use it. Stick with the substr or change the datatype of the column.
Re: Performance Issue [message #623464 is a reply to message #623443] Wed, 10 September 2014 07:14 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
Im getting an error when applying this above code;


Of course, I assumed that hiredate (like in EMP table) is a date not a string.

Re: Performance Issue [message #623470 is a reply to message #623462] Wed, 10 September 2014 07:53 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
cookiemonster wrote on Wed, 10 September 2014 19:00
Don't bother.
Extract is function used on dates (as is trunc). You don't have a date. So there's no point you trying to use it. Stick with the substr or change the datatype of the column.



Extract working for me;

select extract(month from to_date(hiredate,'YYYY-MM-DD')) from emp


but i am trying for trunc which is giving the error.
Re: Performance Issue [message #623472 is a reply to message #623470] Wed, 10 September 2014 08:10 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
trunc fails for exactly the same reason as extract.
But you're missing the point. extract and trunc are not more efficient than substr. They are correct tools to use when working with dates, but you don't have a date. Using trunc and extract in your case will not improve the performance of the query at all.
If substr is the cause of your performance issue then it'll be for the same reason as using any function in a where clause is a performance issue - it stops you using indexes. Converting the varchar to a date and using trunc and extract isn't going to fix that at all.
If the column was a date then there would be ways of writing the query to allow normal index use, but that isn't the case.
That said, as I already pointed out, you can use a function based index.
Re: Performance Issue [message #623474 is a reply to message #623472] Wed, 10 September 2014 08:53 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Function based Index?
Re: Performance Issue [message #623477 is a reply to message #623474] Wed, 10 September 2014 09:03 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Yes, this is time to use the search feature of the documentation.

Re: Performance Issue [message #623479 is a reply to message #623474] Wed, 10 September 2014 09:05 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
stalin4d wrote on Wed, 10 September 2014 06:53
Function based Index?



are both GOOGLE & SEARCH broken for you?
Re: Performance Issue [message #623484 is a reply to message #623435] Wed, 10 September 2014 09:47 Go to previous messageGo to next message
EdStevens
Messages: 1376
Registered: September 2013
Senior Member
stalin4d wrote on Wed, 10 September 2014 05:38
Its Varchar


Arrgh!
And double Arrgh!

see: But I want to store my date as ...

I don't care what "reason" they have, their "reason" is fatally flawed and based on a lack of understanding of basic design principles.
How to Remove Substring Value [message #623856 is a reply to message #623426] Mon, 15 September 2014 08:34 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Hi,

How to remove a substr from this query?

ex. query:

Select empno, hire_date
from Emp 
where substr(hire_date,6,2) != '12';


because the table has more than one 1 crore rows, so we need to remove that
substring for its increased performance.

any idea pls.

Re: How to Remove Substring Value [message #623857 is a reply to message #623856] Mon, 15 September 2014 08:37 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
It has been already suggested to you to change the data type from VARCHAR2 to DATE. Then you won't have to use SUBSTR function at all. As of now, if I remember COM already suggested a workaround to create a function-based index.
Re: How to Remove Substring Value [message #623860 is a reply to message #623857] Mon, 15 September 2014 09:46 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
@OP, the two topics have been merged since they address the same issue as described by you. Please don't open new topic, rather use this topic itself.

You have not provided any feedback to the suggestions given to you above. What did you try so far? Did you implement the suggestions?
Re: How to Remove Substring Value [message #623883 is a reply to message #623860] Mon, 15 September 2014 18:52 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
Actually getting December in every year will require a function based index even if it's date.
Re: How to Remove Substring Value [message #623959 is a reply to message #623883] Tue, 16 September 2014 14:38 Go to previous messageGo to next message
Bill B
Messages: 1971
Registered: December 2004
Senior Member
What version of the database are you using?. you can make a virtual column in versions 11 and 12 where the database would generate a column containing the date value, which you could index and use in your query.
Re: How to Remove Substring Value [message #623988 is a reply to message #623959] Wed, 17 September 2014 02:16 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
A virtual column computed through a function and a function-based index are alike.
Re: How to Remove Substring Value [message #624004 is a reply to message #623988] Wed, 17 September 2014 03:47 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
I didnt tried a function based index, as instructed and strictly adviced by
the client we cant modify the indexes in that table, because it may
affect the production because the same table is used in many programs,
so we are trying at query level.
Re: How to Remove Substring Value [message #624007 is a reply to message #623959] Wed, 17 September 2014 03:55 Go to previous messageGo to next message
stalin4d
Messages: 226
Registered: May 2010
Location: Chennai, Tamil Nadu, Indi...
Senior Member
Its Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
Re: How to Remove Substring Value [message #624012 is a reply to message #624004] Wed, 17 September 2014 04:09 Go to previous messageGo to next message
Lalit Kumar B
Messages: 3174
Registered: May 2013
Location: World Wide on the Web
Senior Member
stalin4d wrote on Wed, 17 September 2014 14:17

so we are trying at query level.


You cannot get rid of design flaws at query level.
Re: How to Remove Substring Value [message #624098 is a reply to message #624012] Wed, 17 September 2014 14:26 Go to previous messageGo to next message
cookiemonster
Messages: 13920
Registered: September 2008
Location: Rainy Manchester
Senior Member
If you can't add indexes then substr is probably as good as it gets.
Substr is pretty efficient, so the only likely ways to speed it up are:
a) use less functions - you'll need to make the column date datatype for that.
b) use function based indexes

Since neither option is available to you you're probably stuck with bad performance.
Re: How to Remove Substring Value [message #624099 is a reply to message #624004] Wed, 17 September 2014 14:29 Go to previous message
Bill B
Messages: 1971
Registered: December 2004
Senior Member
stalin4d wrote on Wed, 17 September 2014 04:47
I didnt tried a function based index, as instructed and strictly adviced by
the client we cant modify the indexes in that table, because it may
affect the production because the same table is used in many programs,
so we are trying at query level.

adding a function based index will NOT alter the other queries unless the query exactly matches your where clause which will make that particular query run faster.
Previous Topic: How to find total count based on date
Next Topic: Update command
Goto Forum:
  


Current Time: Fri Apr 26 20:58:24 CDT 2024