Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.server -> Re: Benchmarks was Re: Which one is better? Oracel 9i or DB2 7.2??

Re: Benchmarks was Re: Which one is better? Oracel 9i or DB2 7.2??

From: Obnoxio The Clown <obnoxio_at_hotmail.com>
Date: Wed, 08 May 2002 16:20:46 GMT
Message-ID: <3cd94ae1.430347807@News.CIS.DFN.DE>


On Wed, 08 May 2002 14:54:23 GMT, Daniel Morgan <dmorgan_at_exesolutions.com> wrote:

>Serge Rielau wrote:
>
>> > There is not a chance I am going to find a TCP report of any value in 90%
>> > of these projects. To me they are just another form of marketing hype.
>>
>> I don't thiok they are mere marketing hype. Yes, they are used for marketing,
>> but the vendors also constantly calibrate their products against them and the
>> benchmarks do comprise some "real world like" queries.
>>
>> Lets' take a look at materialized views (or AST, indexed views...). Oracle and
>> DB2 broke the TPC-D
>> benchmark with those. Were they a benchmark special? No, even Microsoft has
>> introduced them despite not playing in TPC-D (and now TPC-R and -H).
>> Also note that often the availability of the system is 6 months after the test
>> because the test uses bleeding edge technology.
>> I would call TPC a sandbox - simplified? yes - but also relevant.
>> Participating in the game means a vendor is serious about this kind of
>> workloads. Who is on top at any given moment is not really the point as long
>> as they are at the top every so often.
>>
>I can not agree. I think the vendors tweak their products to do well on the test.
>Do the tests emulate systems I build? Unlikely. So if the system being built
>happens to be an exact simulation of the test it might have some relevance.
>Otherwise the test mean little or nothing. And I'm not sure what you mean by
>"serious about this kind of workload." What the vendors are serious about is
>marketshare and profits. They are, after all, for-profit businesses. And if they
>couldn't use these benchmarks for hype their products the benchmarks wouldn't
>exist.
>
>When there is a hacking benchmark where the databases are made available to a
>bunch of 16-22 year olds and there is a security rating I will be interested. When
>the databases are hammered on for months on end until the last one breaks and
>there is a stability benchmark I'll be interested. When they keep adding users
>doing simultaneous inserts/updates/deletes/selects on the same records and the
>same tables and there is a scalability benchmark I'll be interested.
>
>That some query takes 0.0214 seconds vs 0.0235 seconds is not enough to make me
>look up from my latte'.

I beg to differ. The issue is not one of 0.0214 seconds vs 0.0235 seconds. The issue is pushing the envelope and making stuff break. Running arcane and unrealistic benchmarks looking for numbers that a handful of sites around the world actually need makes a better product for us all. These benchmarks stress test the products and provoke the vendors into adding features that can make the products more useful to us lesser mortals (as well as some features that just make the benchmarks run well! :-)

Also, while a TPC benchmark is meaningless as an indicator of absolute performance, the fact that a vendor is playing in the top ten means that performance in the area that the benchmarks cover will generally be acceptable and you can focus on other differentiators, unless perfomance is a particularly key criterion. (This is for those of us playing at the lower end of the market who don't have massive benchmarking budgets or our own R&D departments.)

I would never advocate buying product X, just because it's currently no 1 in the TPC-C. But I would always say that X takes performance seriously if they play the game. Received on Wed May 08 2002 - 11:20:46 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US