Direct I/O on vxfs on EMC

Just curious if anyone's come across this before. Got an email from a friend who's just put an (8i) database with everything (excepting the redo logs) on one large vxfs stripe on top of EMC disks (is standard raid-1 config). The redos are on separate raid-1 (vxfs on EMC) disks.

When the file systems are mounted for direct I/O - all the machine level stats (sar/iostat/vxstat) look fine but the users are up in arms about performance. The only sign of problems is that fibre interface to the disks is showing 1500 io's/sec.

Mount the file systems as per normal (ie without direct io), and users are much happier, the fibre shows 120 io's/sec.

Unfortunately I've just finished at the site - so I can't run any diagnostics. I'm guessing that maybe their Oracle buffer cache is way undersized, or their is some problems with the direct IO on EMC disks (or the EMC cache), but I'd appreciate any other theories...

Cheers
Connor

-- 
==============================
Connor McDonald

http://www.oracledba.co.uk

"Some days you're the pigeon, some days you're the statue..."