Poor RAC inter-instance performance

I'm testing a 2-node Linux RAC installation that's performing poorly under certain circumstances. The cluster is using 1G NICs as interconnect. I already know I'm not supposed to ask for great performance out of it, however the interconnect's capacity is far from reached whenever I notice bad performance.

Here is the behavior:

With only one instance up, any query runs quickly, as expected.

With both instances up, I connect to instance A, run a long query (say, 30-seconds) and it runs normally. At this point I assume instance A has cached some or all the blocks used for the query.

Then I connect to instance B and issue the same query as above. Rows are returned intermittently, up to 4-second delays in between. I assume in this case RAC is trying to ship whichever buffers are found in instance A instead of reading from disk. No DML activity is going on, it's just me and the database.

What confuses me is that I don't see much traffic thru the interconnect, and CPU and disk utilization are far from high. Using vmstat I see increased context switches, but they are just as high as those in the instance that runs fine! I have read note 181489.1 but haven't found much help from it and wonder if it's something else inherent to my architechture (i386).

Thanks in advance for any clues.
Ivan. Received on Mon Mar 29 2004 - 17:11:48 CST