Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.server -> Re: Benchmarks was Re: Which one is better? Oracel 9i or DB2 7.2??

Re: Benchmarks was Re: Which one is better? Oracel 9i or DB2 7.2??

From: blair kenneth adamache <adamache_at_stopspamca.ibm.com>
Date: Wed, 08 May 2002 11:21:58 -0400
Message-ID: <3CD94296.DA2B5609@stopspamca.ibm.com>


But of course in the case of TPC-D, queries had fantastic improvements in response times in each iteration (from dozens of minutes to a few seconds). And, the features that accomplished are used by lots of customers with non TPC-D like workloads. Add to that the fact that one of the TPC-D queries would cause a particular RDBMS to crash when the workload was first published in 1995, and I think we have a useful workload. If these benchmarks were used only for hype, you might have a point. But TPC-D, TPC-C, TPC-H and TPC-W are all run internally by RDBMS vendors to gauge their performance quality.

Daniel Morgan wrote:

> Serge Rielau wrote:
>
> > > There is not a chance I am going to find a TCP report of any value in 90%
> > > of these projects. To me they are just another form of marketing hype.
> >
> > I don't thiok they are mere marketing hype. Yes, they are used for marketing,
> > but the vendors also constantly calibrate their products against them and the
> > benchmarks do comprise some "real world like" queries.
> >
> > Lets' take a look at materialized views (or AST, indexed views...). Oracle and
> > DB2 broke the TPC-D
> > benchmark with those. Were they a benchmark special? No, even Microsoft has
> > introduced them despite not playing in TPC-D (and now TPC-R and -H).
> > Also note that often the availability of the system is 6 months after the test
> > because the test uses bleeding edge technology.
> > I would call TPC a sandbox - simplified? yes - but also relevant.
> > Participating in the game means a vendor is serious about this kind of
> > workloads. Who is on top at any given moment is not really the point as long
> > as they are at the top every so often.
> >
> > Just my 2 cents
> > Serge
> > --
> > Serge Rielau
> > DB2 UDB SQL Compiler Development
> > IBM Software Lab, Canada
>
> I can not agree. I think the vendors tweak their products to do well on the test.
> Do the tests emulate systems I build? Unlikely. So if the system being built
> happens to be an exact simulation of the test it might have some relevance.
> Otherwise the test mean little or nothing. And I'm not sure what you mean by
> "serious about this kind of workload." What the vendors are serious about is
> marketshare and profits. They are, after all, for-profit businesses. And if they
> couldn't use these benchmarks for hype their products the benchmarks wouldn't
> exist.
>
> When there is a hacking benchmark where the databases are made available to a
> bunch of 16-22 year olds and there is a security rating I will be interested. When
> the databases are hammered on for months on end until the last one breaks and
> there is a stability benchmark I'll be interested. When they keep adding users
> doing simultaneous inserts/updates/deletes/selects on the same records and the
> same tables and there is a scalability benchmark I'll be interested.
>
> That some query takes 0.0214 seconds vs 0.0235 seconds is not enough to make me
> look up from my latte'.
>
> Daniel Morgan
Received on Wed May 08 2002 - 10:21:58 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US