Re: Benchmarks was Re: Which one is better? Oracel 9i or DB2 7.2??

"Sailesh Krishnamurthy" <seesignature_at_cs.berkeley.edu> wrote in message news:mjqsn52mypb.fsf_at_localhost.localdomain...

> >>>>> "Daniel" == Daniel Morgan <dmorgan_at_exesolutions.com> writes:
>
> The thing is, vendors assemble special benchmark teams comprising of
> people who have actually designed, architected and implemented
> significant parts of the engine. Ergo, they really understand the
> product and can tune it very well - way beyond the ability of the
> average user. This is what makes a benchmark unrealistic.

At first, I disagreed with the above but as I was penning my initial response, I realize that you're right, it's beyond the ability of the average user. I have worked on a TPC-C (published) and one of my friends is still on the DB Perf team and continues to work at publishing numbers. The 'team' is not as big as folks might think but it does involve a lot of folks:

• DBMS vendor
  • provides kit
  • engineering resources to Q&A
• hardware vendor:
  • DB engr (with a definite hardware slant and a DB slant too of course - CS degree needed! :)
  • kernel hacker
  • file system hacker

the combined knowledge is _huge_!!!!

Another benefit of the TPC, is the information gleaned:

• performance characteristics of the hardware:
  • disk
  • scalability of internal structures
  • controllers
  • CPU's
• perf characteristics of the software
  • the 'knee' on the curve (not necessarily application specific)
  • scalability

The people I worked with benched DB2, Sybase, Oracle and Informix. Before that, I was very Sybase-centric and after I left the team I was and am DB-agnostic.

The other salient point was that as a member of the DB Perf team, I not only worked in the lab but also had to apply the knowledge learned out with some key customers. Customers that had paid oodles of cash for their hardware. I was flown out several times at a moments notice to 'solve the problem' The knight in shining armor syndrom got old _very_ quickly.

--
Pablo Sanchez, High-Performance Database Engineering
mailto:pablo_at_hpdbe.com
http://www.hpdbe.com
Available for short-term and long-term contracts