Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.server -> Re: Optimal degree of parallelism (long)

Re: Optimal degree of parallelism (long)

From: Noons <wizofoz2k_at_yahoo.com.au>
Date: Wed, 3 Sep 2003 21:08:30 +1000
Message-ID: <3f55cd4c$0$14564$afc38c87@news.optusnet.com.au>


"Saikat Chakraborty" <saikatchak_at_hotmail.com> wrote in message news:2b82e9b726801a8b6cb11d2835051ae2.16981_at_mygate.mailgate.org...

> Great! that proves my observation. I understand in Unix systems,
> each process must give up the CPU for other process between cycles.

Not just that. Unix flavours differ somewhat, but the original spec from AT&T System V provided for multiple priority levels with strict time-sharing for all processes within the same priority level. That meant a strict round-robbin execution on a time-slice. The time-slice was a tunable parameter, so was the number of priority levels. Of course, if a process initiated a wait-on operation, it lost its time-slice even if it had not yet expired.

Between priorities, the only way a lower priority (which was a higher number!) process could execute was if all processes in all higher priority levels were either waiting (on I/O, semaphore or timer). When a process of a higher priority became ready the scheduler could (optional configuration) pre-empt a lower priority time-slice and resume execution of the ready higher priority. Or it could wait for the end of the time-slice and get the CPU on the next scheduler allocation.

With time, processes on higher priorities would gravitate to lower ones as their CPU usage increased. If they kept doing a lot of waits, they'd stay at the same priority level or even increase. The theory being that CPU-bound processes should be managed with time-slices, while interrupt-bound programs should get CPU when needed in small chunks. Again, the thresholds for all this were tunable parameters.

This hasn't really changed much along the years. HP/UX had a slight variation for real-time priority levels. This basically meant processes in real-time had a higher priority than OS interrupt post-processing tasks. Interrupt servicing tasks being traditionally the highest in the OS. I believe there is some work along these lines for Linux as well. But otherwise it inherited the default scheduler from vanilla Unix.

Windoze server works very much along the same lines. But you can't really tune anything on the scheduler as everything is supposdely "self-tuning". Which means essentially it can be very good *if* you have a lab system.

> In addition, once I ran a PQ on a 64 processer machine (Compaq GS 320)
> and got carried way.
> I used a degree or parallelism of 64 on a 80 million table. The
> performance
> dropped sharply. That shows even a high degree of PQ slaves with enough
> resource is not good enough.

Can be bad. Like any piece of code, the OS scheduler can become overloaded. This used to be called "thrashing". It basically means it goes into a race condition where it spends more time sorting through what it needs to do next than actually doing it. With dire results.

Of course, increasing the PQ degree beyond a sane level can help this situation. The trick is to keep an eye on the CPU wait queue lengths. Another thing to watch out for: the queue lengths indicate different types of problems depending if you run a single CPU or a 64 CPU SMP system. Queue length is usually more reliable to indicate thrashing on single or small number of CPUs. Not all SMP systems are really pure SMP. Numa-Q and other "cluster" systems are notably different. It is a fascinating field. There are two books that are superb in analyzing all this in the Oracle context.

> About my first question, the peer pointed out that from a book from
> Oracle
> press by an author with a 'D' starting his first name. Therefore I told
> him
> to stop believing him <g>

There you go. To be entirely fair, we have to determine if that was taken out of context from the book or "adapted" to later versions when the book related to earlier ones. It's only too easy to brand a given author as "bad" yet the stuff made sense back when the book was written.

Harrison's first edition on tuning SQL might have a few incorrect things if taken in the context of 10g. It was written with 7.3 in mind. But I wouldn't derive from that anything from Harrison is bad: his second edition is as good as it gets. And the first edition was too, *when* it came out.

It's all relative, I suppose. There are of course books that are just plain bad or not useful at all. In any edition.

--
Cheers
Nuno Souto
wizofoz2k_at_yahoo.com.au.nospam
Received on Wed Sep 03 2003 - 06:08:30 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US