Re: performance issues on sun

The Solaris kernel has to have asynchronous IO enabled. If you are running with at least Solaris 2.8, this should not be a problem. If you have your files on VxFS, then I would VERY strongly suggest looking into Veritas DB Edition, particularly, Quick IO (writes) and cached qio (reads). This gives you true DIRECT IO on cooked journalled file systems (VxFS). I have seen performance gains on the IO of up to 400% just from turning this on. Moreover, if you are looking at waits, where I installed this at a previous client, I saw my idle CPU time go from 0% - 5% range, into the 70% range, and the iowait% reduce on average from 50% - 90% range to single-digit figures. Also the load on the machine was greatly reduced. There was also a management issue . See if you can understand this logic : They were using an A1000 with RAID5 and 8 MB write cache (I told them how RAID5 hurts redo log writes, TEMP and RBS writes until I was blue in the face, but they kept showing me that RAID5 allowed them to configure more logical space than RAID 10, duh !) with wait for it .... SCSI UW2, yep 2, that means a full 40MBPS throughput, woohoo !. So they were willing to fork out the 30 grand it cost for Qio, than to replace the A1000 with a new Adaptec Durastor 7220 SS which would have given fibre speed, and about 4 times the amount of logical space, and they would have gotten change from 20 grand, and there was no annual software license maintenance fee. Don't get me wrong, I think Veritas has some FANTASTIC products, a lot is dependent on the support person they assign to your account (it took me about 10 support calls to realize that the reason my reads were not going any faster was because I needed to configure cached-quick-IO, which was NOT in any of the marketing stuff, only upon re-reading the technical guide for the 4th time, did I spot the 3 line entry about it and decided to ask questions.

Prior to Solaris 2.8, asynch IO on Solaris was not considered safe (by the SA's anyway), but as of 2.8, one can enable asycnh IO on Solaris for cooked file systems, if you can convince the SA that Oracle has its own backup and recovery mechanism. Then the next thing you need to worry about is stripe size and getting it just right. Oh, and if you can ditch RAID5 in favour of RAID 10, please do so as early as possible. I have just been reading through my complimentary copy of Gaja and Kirti's book, and Gaja does a great job of describing stripe sizes (Gaja, you did not mention cached quick IO only QIO, tut tut ! ). I also disagree with Gaja about the folklore on HAVING to separate indexes and their tables into separate tablespaces, that depends on where the volumes are physically mapped to and unless you can see this information, there is no basis for making this claim either way. But that is another story, for another thread.

Hope this has helped you out. Regards :

Ferenc Mantfeld
The pain of regret is far worse than the pain of discipline !. ----- Original Message -----
To: "Multiple recipients of list ORACLE-L" <ORACLE-L_at_fatcity.com> Sent: Wednesday, February 26, 2003 1:11 AM

> All
>
> We are attempting to move some applications off Compaq T64 into Sun
Solaris
> 8 and running into performance issues.
>
> I am trying to rebuild an index which is taking more than 3 1/2 hours
while
> it used to take < 20 min on T64.
>
> I find most of the waits on DIRECT PATH READS and DIRECT PATH WRITES. The
> index tablespace and the temporary tablespace are on separate mountpoints
> which reside on separate disks.
>
> I am doing a "truss" on the session and see that its doing the following
>
> kaio(AIOWAIT, 0xFFFFFFFFFFFFFFFF) Err#22 EINVAL
> pread(364, "\b02\0\0\v\099E1 f h ECB".., 1048576, 0x26784000) = 1048576
> kaio(AIOWAIT, 0xFFFFFFFFFFFFFFFF) Err#22 EINVAL
> lwp_cond_wait(0xFFFFFFFF7CED7F70, 0xFFFFFFFF7CED7F80, 0x00000000) = 0
> pwrite(408, "0602\0\0\nC41007 f h SDD".., 49152, 0x10401C000) = 49152
> pwrite(408, "0602\0\0\nC410\n f h SDD".., 49152, 0x104028000) = 49152
> pwrite(408, "0602\0\0\nC410\r f h SDD".., 49152, 0x104034000) = 49152
> pwrite(408, "0602\0\0\nC41010 f h SDD".., 49152, 0x104040000) = 49152
> pwrite(408, "0602\0\0\nC41013 f h SDD".., 49152, 0x10404C000) = 49152
> pwrite(408, "0602\0\0\nC41016 f h SDD".., 49152, 0x104058000) = 49152
> pwrite(408, "0602\0\0\nC41019 f h SDD".., 49152, 0x104064000) = 49152
> pwrite(408, "0602\0\0\nC4101C f h SDD".., 49152, 0x104070000) = 49152
> pwrite(408, "0602\0\0\nC4101F f h SDD".., 49152, 0x10407C000) = 49152
> pwrite(408, "0602\0\0\nC410 " f h SDD".., 49152, 0x104088000) = 49152
> pwrite(408, "0602\0\0\nC410 % f h SDD".., 49152, 0x104094000) = 49152
> pwrite(408, "0602\0\0\nC410 ( f h SDD".., 49152, 0x1040A0000) = 49152
> pwrite(408, "0602\0\0\nC410 + f h SDD".., 49152, 0x1040AC000) = 49152
> pwrite(408, "0602\0\0\nC410 . f h SDD".., 49152, 0x1040B8000) = 49152
> pwrite(408, "0602\0\0\nC410 1 f h SDD".., 49152, 0x1040C4000) = 49152
> pwrite(408, "0602\0\0\nC410 4 f h SDD".., 49152, 0x1040D0000) = 49152
> fdsync(408, O_RDONLY|O_SYNC) = 0
> pwrite(408, "0602\0\0\nC410 7 f h SDE".., 49152, 0x1040DC000) = 49152
> pwrite(408, "0602\0\0\nC410 : f h SDE".., 49152, 0x1040E8000) = 49152
> pwrite(408, "0602\0\0\nC410 = f h SDE".., 49152, 0x1040F4000) = 49152
> pwrite(408, "0602\0\0\nC410 @ f h SDE".., 49152, 0x104100000) = 49152
> pwrite(408, "0602\0\0\nC410 C f h SDE".., 49152, 0x10410C000) = 49152
> lwp_cond_wait(0xFFFFFFFF7CF0DF70, 0xFFFFFFFF7CF0DF80, 0x00000000) = 0
> lwp_cond_signal(0xFFFFFFFF7CF0DF70) = 0
> pread(364, "\b02\0\0\v\09A ! f h ECB".., 16384, 0x26884000) = 16384
>
> I think it is trying to do a KAIO call and failing. Then it attempts a
> synchronous PWRITE call.
>
> But our SAs are not able to help us to confirm this. Have any of you seen
> this issue?
>
> Any inputs would be greatly appreciated. I'll gladly provide you with addl
> info if you need.
>
>
> Thanks in advance
>
> Babu
>
> _____________
> This e-mail transmission and any attachments to it are intended solely for
> the use of the individual or entity to whom it is addressed and may
contain
> confidential and privileged information. If you are not the intended
> recipient, your use, forwarding, printing, storing, disseminating,
> distribution, or copying of this communication is prohibited. If you
> received this communication in error, please notify the sender immediately
> by replying to this message and delete it from your computer.
>
>
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.net
> --
> Author:
> INET: babu.nagarajan_at_Cummins.com
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).
>
>

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.net
-- 
Author: Ferenc Mantfeld
  INET: mantfield_at_connexus.net.au

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).