Re: Discreet transactions?

Adam Green (adamg_at_informix.com) wrote:
: At the very least, the current crop of TPC-C results, unlike the
: TPC-A tests, are all but universally accepted as sound and
: can be applied to real world situations in a fair and reasonable
: manner without the risks of the gross distortions found in the TPC-A
: results.
 

: As for running your own bench tests? Seriously?

My own observations are that the TPC benchmarks are not any better than any other TPC benchmark in forcasting expected performance. I do not know of any universal agreement concerning the TPC-C benchmark other than at Informix who so far is the only company to run it. David McGoveran, a very respected analyst in this area, echos my views of the TPC and has documented his views in the Network Computing article which I mentioned.

My clients do their own benchmarks all of the time and not surprisingly their results are no where near as optimistic as the TPC results. The essential issue is that it is the RBDMS vendors who created the benchmarks and the TPC specifications do a very good job of reflecting the origins of its funding. I
reaize that RDBMS vendors spend hundreds of thousands of dollars promoting the TPC benchmark - primarily because it helps yield millions in sales - but it does not provide customers any useful information. If in house benchmarks are not feasible then a customer should inquire what other customers are doing. In my surveys I have found that the vast majority of the Unix RDBMS applications (probably over 95%) are running at less than one transaction/second.

-- 
Richard Finkelstein                        Voice: 312-549-8325     
Performance Computing, Inc.                Fax: 312-549-4824 
Chicago, IL                                Mail: rfinkel_at_infochi.com