Re: Linux betas NT in TPC testing, running Oracle8

In article <7fo412$9nf$1_at_son-of-bofh.net>, shimpei+usenet+.mil+.gov_at_BOFH.submm.caltech.edu says...
> Stephen Edwards <ja207030_at_primenet.com> writes:
> >: It's sponsored by more than one company (unlike the Mindcraft study),
> >: but I have to wonder if maybe Oracle deliberately cripples their NT
> >: version somehow.
> >
> >The test is completely bogus. But I'm not suprised in the slightest.
> >
> >"No specific operating system tuning was done, all the default NT services
> >were running, with the default priorities and system parameters."
> >
> >Oh no... not biased in the slightest.</SARCASM>
>
> Biased, no. Bogus, yes.
>
> At least it's a fair benchmark in that they tested both operating
> systems "out of the box," giving both machines equal attention (i.e.
> none), whereas the Mindcraft benchmark was unfair in that NT got a lot
> more tuning attention than Linux did. However, it is completely
> useless as a real-world benchmark because anyone who runs Oracle who
> cares one iota about performance is going to perform *some* amount of
> tuning, and get completely different results. At least the Mindcraft
> survey is useful reading if you run NT servers and want to know how to
> tune them up to the hilt.
>
> --
> Shimpei Yamashita <http://www.submm.caltech.edu/%7Eshimpei/>
>
>

Good point. The two most important factors in any benchamrk comparision are how fair and how representative of the real world it is. Running both OS's untuned is unbiased, but hardly representative of how people in the real world would run Oracle. Also, weren't the databases in the test very small?

The benchmark I'd like to see is the following: Some independent group sets up the hardware, including the networking, in a way that is representative of an Oracle installation. They also design the database layout, the benchmark, etc. Then, MS or someone else with a vested interest in NT winning, gets to tune the software however they want. Tests are run, then the Linux crowd gets their turn, with an equal chance to tune only the software. Both sides are allowed only commercially available products (including installation media), and all tuning changes are recorded and published with the results.

This would still likely result in people from one camp or another complaining about how the specific hardware favored the other side (possibly with merit, mossibly not), but it would probably give us a much more meaningful result. It would be a lot of fun, too. <g>

Lou Received on Thu Apr 22 1999 - 23:50:21 CEST