Re: The old raw devices chestnut.

Paul Watson wrote:

[ ... snip ... ]
>
> But if the cache is too small or being turned over very quickly the
> cache will be slower 'cos you to copy from disk to cache and then
> copy from cache to the app.
>
> Interestingly on the bigger Sun servers the absolute bandwidth to
> disk is significantly larger than the bandwidth to memory so, in
> theory, you can the data from disk faster than from memory - I remain
> unconvinced:-)
>

[ ... snip ... ]

[ not cross posted as it is IDS centric ]

small cache is always a bad thing, but slow CPUs are also a bad thing, getting more important recently

The problem I've seen too often now looks like this: Marketing & brochure, paperware & vaporware tells you 'We have it for 10 times the money'
so you tend to believe it is at least the same thing you could get for a tenth of the $$$ elsewhere.

BUT: do a simple test to find out the access rate to LRU BUFFERS per second on each and any of the big iron machines
(like SELECT '1' from <table with x pages size>) and choose size so, that you have x + 10% BUFFERS configured & and let the whole thing output to /dev/null.

Now test this repeatedly & see that you have 0 dskreads on second or later tests.

Then look at the figures :(

To make a long story very short:
If you have CPUs so slow, that one must be happy with 35-40K buffreads per second, then it is easy to be fast on today's disks (or, if your specific SMP machine vendor has SCSI 320 only in Q4/04 or later *hint, hint* - it is more appropriate to speak about yesterdays disks ;)
because your limit is in fact the lookup speed in the LRU BUFFERS, no matter if you have to fetch them first from disk or not :(

The contrary is true if you are on modern CPUs using high bandwith into memory (like seen on a 4 x 200 Mhz southbridge - aka 800 Mhz southbridge).

It is true even if you avoid 'consolidated storage' aka SAN and go for 16 way parallel JBODS or RAID 10 -- no RAID 5 -- no RAID 5 -- remains true, *very* true (the more SAN the more true it remains - see Art's postings for the nasty details of RAID 5, and in reality SAN = RAID 5)

Of course the ordinary disclaimers must follow YMMV
YMNNTPISO (You may not need the performance I speak of) YAIAJS (You are in a Java shop)
and so on.

I do know, that even on a quad Opteron system with hyperthread option (for which we do *not* have IDS at this very moment) you cannot run a workload of more than 5K users, if the application software is not designed to scale well onto a grid .....

Let me end with another sentence, which is maybe too true and thus may cause some pain somewhere:

If your disks are RAID 5 or SAN or slow (which is all the same in my humble point of view) and you therefore do not have to think about placing or you cannot test if raw access is faster than cooked (because you only have 1 SAN which is for production) I can affirm that is does not matter, if using cooked files or raw. Slow is slow. 10-20% faster than slow is still slow.

If you have the luxury of beeing able to test the difference, then TEST IT!
Suddenly you will find out, why the SAN vendor tries to sell software into your shop, which is able to reduce the very nasty effcts of hot spots in your SAN...

If you are on fast disks, then raw is faster. If you are on *fat* disks caches then raw is faster. (fat means like 7-9% of total disk size) If you are on solid state disks, raw is faster. How much faster depends on many details. I never saw a difference smaller than 10% faster. I saw drivers where the 15% difference comes from the CPU load of the fs cache algoriths. I can affirm that the difference in read speed can go up steeply if IDS SW mirroring is in use, because read parallelism comes into play at full gain. It depends, but raw really pays off, ususally.  

Better stop my rant now, and sorry if I did hurt any markting soul's feelings too much here.

dic_k

(Everyone is invited to hire me for doing some benchmarking on your very system)

-- 
Richard Kofler
SOLID STATE EDV
Dienstleistungen GmbH
Vienna/Austria/Europe