| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
 |
 | |||
| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
|
| |||
Home -> Community -> Usenet -> c.d.o.server -> Re: Server Databases Clash
This is posted simply in the interest of completeness. It in no way reflects
a personal opinion. Microsoft's official response is:
· Microsoft believes in public, independent benchmarks. We have demonstrated our strong support for independent benchmarks through our participation-with other hardware and software companies-in the Transaction Processing Performance Council (<http://www.tpc.org>).
· We have had a very good history working with ZD Labs on many past benchmarks. In this case, however, we found that many of the architectural decisions ZD Labs made weren't well suited to our platform. Despite having optimized source code available on the Web already (<http://www.gotdotnet.com/compare>), ZD Labs refused to use the version of the benchmark that would reflect how an actual Windows developer would build their application.
· We believe that by not involving Microsoft or other vendors in the architectural decisions up front, ZD has not built a benchmark that accurately reflects real world design and coding practices. These benchmark results are unlikely to reflect how the platforms will really perform. We believe that ZD is doing a huge disservice to its readers because its tests are done in a vacuum and the results are thus questionable.
· Even if SQL Server and/or .NET do well in ZD's tests, we object to the manner in which the tests were conducted. Specifically:
o We are disappointed they did not give us chance to review their code or provide input on their methodology despite repeated inquiries prior to the testing.
o We believe the app tested was not properly optimized for .NET; it was a straight port of a Java app and is not coded to .NET best practices and not representative of .NET performance.
o We do not believe that ZD's design or coding reflect real world practices that customers use. As a result, we would strongly caution anyone considering using these results to make a platform or database choice.
o The tests only measure the performance of BEA with each particular database, they do not provide reliable metrics to evaluate database performance outside of the BEA app server. (I.e. ZD Labs cannot conclude from their testing that database x is faster than database y, only that BEA is faster with database x than with database y.)
o ZD may not publish the code used for these benchmarks; they are still debating that point. For the public to rely on benchmarks, they must be reproducible (see the full disclosure documents for any benchmark at <http://www.tpc.org> for an example of proper documentation-including source code-for benchmark reproduction).
· Microsoft's existing results (for .NET vs. J2EE with the same application tested by ZD Labs, the Nile Web bookstore) are public, fully documented and provided with source code. Our Nile performance benchmarks are published at <http://www.gotdotnet.com/team/compare/nileperf.aspx> . We believe that these results are more indicative of how .NET will perform when actual customers use it. This belief is underscored by many users who are seeing huge performance gains on .NET. Additionally, we believe the database benchmarks of the TPC (<http://www.tpc.org> are the most reliable method of comparing RDBMS scalability and performance for a variety of workloads.
"cooper" <jcheong_at_cooper.com.hk> wrote in message
news:5ea01b36.0203070813.7d28e04c_at_posting.google.com...
> Database benchmark results tested on the same hardware
>
> http://www.eweek.com/article/0,3658,s=708&a=23115,00.asp
Received on Thu Mar 07 2002 - 13:41:54 CST
 |
 |