Re: lies damn lies and benchmarks

Andrew Mobbs <andrewm_at_chiark.greenend.org.uk> wrote in message news:<Efb*2tEnp_at_news.chiark.greenend.org.uk>...
> Mike Ault <mikerault_at_earthlink.net> wrote:
> >
> >Looking at the most current top ten values overall Oracle achieves a
> >TPCC/CPU-MGHTZ rating of 8.11 beaten only by Symphonies 8.24 (whatever
> >the heck Symphony is) SQL2000 comes in 4th place at 3.36. The most
> >bang for the buck comes in with Oracle's 1,019,668.87 per
> >TPCC/CPU-Mghrtz against SQL2000 at 1,592,560.56. UDB comes in at 4.92
> >and a cost of 1,733,687.45. UDB beats SQL2000 in TPCC/CPU-Mghtz but
> >looses in overall price. Oracle beats both handling when you remove
> >the obscuring junk in the numbers.
>
> Who cares about TpmC/MHz ? That's a meaningless metric. CPU performance
> is only loosely correlated with the clock speed, if you look at the SPEC
> CPU benchmarks, you can find a 750MHz PA-RISC outperforming a 1.5GHz
> Pentium 4.
>
> Oracle is a great database, with many impressive and useful
> features. TPC-C shows it scaling to impressive levels, but it (and
> Unix vendors) fall down dramatically on price/performance in the
> mid-range. Unless Oracle and HP/IBM/Sun want to give up this ground to
> MS, they better do something to fix this.
>
> For example, compare these two results:
> http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=102031101
> http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=102012901
>
> These show two benchmarks both published earlier this year, with an
> IBM/SQL Server system outperforming an Compaq/Oracle system that cost
> twice as much.

Most of Microsoft's TPCC numbers are meaningless since they represent a system no one in their correct mind would build and try to maintain, come on now, a 32 node 4 cpu/node federated database? Who are they kidding. Don't forget cost to maintain those 32 servers and their associated disk farms as well as the time required for the additional programming support to maintain all the 2PC logic not to mention rebuilding the whole thing when adding or removing nodes and any other number of maintenance tasks. Cheaper is only cheaper if it is reliable and requires less maintenance.

When comparing costs, also compare features. For the same feature set Oracle is comparable in price (standard Oracle is the same as SQL2000, not enterprise Oracle as most comparisons from MS land try to fob off on you.) I suggest looking at the Database shootout done by PC Magazine (which is notorious for being in the Microsoft camp) who did they choose as their editors choice? Oracle! the same article has a detailed feature comparison spread sheet for all of the major databases (as well as the public domains such as MYSQL.)

My ratio is as good as any other metric in that it tries to sift through the haze generated by such artificial constructs as a 32 node federated database. I would much rather have Oracle's TPCC performance on a single, maintainable 64 CPU box going against an industry standard SAN or disk farm than an equivilent of Microsoft's on 16-4 cpu windows boxes each with their own disk array using some god awful federated architecture. Even assuming a multiplicaiton factor that is directly proportional to the number of nodes this means for a 64 CPU setup (16-4 cpu nodes) Microsoft would only achieve a TPCC of around 206,000, 50% less than Oracle's.

Mike

Again, do your homework. Received on Tue May 07 2002 - 11:46:14 CDT