Sun/Oracle server growth: more CPUs or additional servers?

We expect our Oracle server (Sun 1000 with two CPUs running Solaris) to grow in the next couple years. We aren't sure, however, if we should be adding additional CPUs (and memory and disk space) to our current box, or  if we should instead buy additional servers.

-How well does Oracle scale on a multiple CPU system like Sun's? Can you
max out the server with CPUs and memory and continue to see better performance? Or does performance plateau at some point?

-Is it a sound strategy to keep all your eggs in one basket (i.e., one
big server, instead of multiple interconnected servers)?

-What performance issues are there with multiple servers? For instance,
what if I want to run a query that joins tables on different servers?

Any thoughts, opinions, or real-life experiences would be greatly appreciated.

Thanks in advance.

Steve Miles
Seattle Water Received on Thu Apr 04 1996 - 00:00:00 CEST