Re: Linux betas NT in TPC testing, running Oracle8

r.e.ballard_at_usa.net wrote in message <7hatok$79i$1_at_nnrp1.deja.com>...
>In article <V8#ifV9m#GA.181_at_pet.hiwaay.net>,
> "nik" <ndsimpso_at_ingr.com> wrote:
>>
>> r.e.ballard_at_usa.net wrote in message <7guvu2$bi9$1_at_nnrp1.deja.com>...
>> >In article <m1emkwpwy5.fsf_at_inconnu.isu.edu>,
>> > Craig Kelley <ink_at_inconnu.isu.edu> wrote:
>> >> r.e.ballard_at_usa.net writes:
>> >> [snip]
>> >>
>> >> > It isn't a true "conspiracy", but the folks who audit these
>results
>> >> > cannot accept the Linux terms as legitimate results. If one were
>to
>> >> > consider only the raw hardware costs - which could be
>competitively
>> >> > obtained for $50k-$60k and the software costs which are in the
>$2k
>> >> > range, and the mainenance contracts available from companies like
>> >> > Flagship ($6,000-$20,000 for 5 years) this is typical of Linux
>> >> > "bargain basement" environments, a total of $80k. If Linux were
>> >> > able to crank out 8,000 TPC/M (plausable when you compare the SCO
>> >> > numbers), then Linux would still be in the $10/TPC range. Just
>> >> > looking at one of the "low-end" NT machines, it's easy to see how
>> >> > Linux could generate some rediculously low $/TPC numbers. NT
>> >> > generates $30/TPC with it's bottom of the line systems.
>>
>> NT systems are in low 20s for 4-way XEON, no one bothers to benchmark
>the
>> 2-way PIII systems in TPC/C because they run out of memory before they
>run
>> out of other horsepower, this would apply to LINUX as well.
>
>It really depends on what you are trying to accomplish. If you want to
>get a 7000 TPM benchmark, you need a big bazooka. If you want 2000 TPM
>and get a low $/TPM rate, you can use a smaller box.
>

This is equally true for NT, NT boxes were producing 2K results half a decade ago, and since the hardware requirements would be very simialr (hint, much of the hardware requirements for TPC/C are netwrok hubs and the disk subsystems and most of the software cost is in the database) the difference in cost of the OS would be lost in the noise, so I sincerely doubt you would see any significant price performance difference in a system configured to produce 2K TpMC regardless of the OS.

>> >> And in this day-and-age, benchmarks are becoming worthless. If
>they
>> >> were so important, nobody would even use Windows NT, Microsoft's
>SQL
>> >> Server or MySQL. People want *reasonable* solutions to their
>> >> problems, both in terms of performance and price. Linux only needs
>to
>> >> meet this requirement in order to satisfy the majority.
>> >
>> Thousands companies use SQL Server for database
>> applications, so a benchmark
>> using SQL Server is very definitely
>> of interest to customers purchasing
>> database systems. A 4-way XEON server
>> running NT and SQLServer is pushing
>> the 25K TPM under the TPC/C benchamrk,
>> this would easily meet the database
>> requirements of hundreds of everyday applications.
>
>But at the same time, a 6k TPM benchmark result on a single processor
>Linux engine is not even publishable.

Because the rules of the benchmark require you to factor in support costs that today can't be bought for a LINUX based system. This is a problem of the benchmark requirement which are set by the TPC council, the majority of whose members come from the database companies and UNIX vendors.

>Even if a P2/400 could only
>crank 8k, that would still leave the cost of dual systems (the
>alternative to a single system with support contract) it under $16k
>giving a benchmark of $2/TPM.

Adding a second 400MHz processor to one of the servers I sell would add roughly $300 to the price of the system, hardly likely to impact price performance noticeably. Also I very much doubt that a single 400MHz could come close to 8K transactions on TPC/C, and there is no way that the hardware would come in under 16K. The disk space required for a 5.3K result was 75x4GB drives, an 8K result would require more disk space (simply because the database size scales with performance.) Lets assume, for the sake of argument that it 1.5x the space and we are using 9GB drives, you still need ~50x9GB drives at say $450 each, thats 22.5K just for the drives, and we haven't included disk housing, controllers etc etc.

>
>Until actual verified legitimate numbers are published, Linux $/TPM are
>up for grabs. You can get some pretty cheap hard drives, some pretty
>cheap transaction monitors, and some pretty cheap LAN cards and create
>a pretty incredible system for very little money. Back in the days when
>you needed 10 drives to get a 20 gig database, it was a challenge.
>Today, with 16gig drives, RAID in software, and cheap DIMM memory, it's
>not that hard to come up with some respectable numbers.
>
On TPC/C, it's a lot harder than you think. TPC/C is a small I/O benchmark, you seem to be indicating that EIDE drives would be suitable (judging by the choice of 16GB drive size) if so, think again. If you are trying to say that TPC/C doesn't reflect the database needs of the vast majority of database applications, then I'd agree completely. I'ts designed as big-iron benchmark for vendors to use in benchmark pissing contests.

>> > Web Server Benchmarks - Linux carries as
>> > much as 8 times the capacity
>> > of Windows NT. At very low levels, NT gives
>> > slightly better response times, but Linux/Apache
>> > response time is nearly flat or linear while NT
>> > deteriorates at an exponential rate.
>> >
>>
>> Other than you're opinion, what actual real data
>> like SPECWeb results do you
>> have to back this claim up?
>
>I had some in the archives. I'll look them up.

I wait with baited breath.

>> > Availability - NT Availability has improved
>> > from 95% in 1996 to nearly 99.7% in 1999.
>> > Linux has gone from 99.98% to 99.998% this means
>> > Linux is down for about 5 minutes every 3 months.
>>
>> Show me a singgle company anywhere in the world
>> that will guarnatee 99.998%
>> uptime for a LINUX server. This is completely bogus claim.
>
>Neither is a guarantee. Both are simply the observations obtained
>from organizations with numerous servers. The NT configuration was
>based on an Insurance company using and keeping records on over 2200
>servers. The Linux numbers came from a number of sources including
>Dejanews and some ISPs.

But without knowing the complexity of the applications and the loads on the servers, this data is meaningless, for example, a LINUX box doing simple IP routing on T1 link is a very different kettle of fish to an NT box handling a large database with hundreds of transactions/sec.

>
>> > NT is down 5 minutes per week. In
>> >response to Linux stability, Commercial UNIX systems
>> >are targeting 5 minutes/year.
>>
>> Hmm, seems COMPAQ, HP and UNISYS would all disagree
>> with you about the uptime of NT, they all have packages
>> which guarantee (and will pay you money
>> if they fail to meet the guarantee) uptime
>> on NT configurations well in
>> excess of what you are quoting.
>
>My understanding was that they could guarantee certain clusters of
>certain configurations up to 99.97%
>One week equals 10080 minutes (7*24*60). And 99.97% of 10080 is
>10077 - leaving a difference of 3 Minutes. This is about the time it
>takes to reboot an NT server. With multiple kosher configurations in
>SMP configurations and RAID drives and a gig of RAM, you can get a much
>better rating. Furthermore, scheduled outages are not included in that
>downtime calculation and each machine is to be rebooted at least once a
>week (I believe the contracts call for daily reboots).
>
>This is significantly different from Linux systems that run online
>housekeeping without down-time.

Again doing what?

>Let's see. Which is so funny - that, in order to have a
>secure, reliable, high performance system you need to have
>a separate fire-wall (A UNIX system) a separate router (A
>CISCO/UNIX system), and a separate machines for DNS, IIS,
>and SQL Server. You could throw all of this into a single
>machine but NT performance rapidly deteriorates because
>the context switching forces it to flush the cache so much
>that the performance deteriorates horribly. I defer to statements
>made by Gartner Reports who claim that it takes 5 NTs to do the
>work of one Linux or UNIX machine.

I have a large ISP customer in South Alabama (roughly 15K users today) that runs mail, DNS, web hosting, news, chat, a firewall, inbound/outbound virus checking and sundry other services on four NT boxes. Hardly an outlandish amount of hardware for the requirements of the job.

>
>Perhaps you dispute my claim that all of these functions can be
>provided on a single machine. Many users of _at_home, the cable-modem
>system now owned by AT&T, use Linux machines because they share a
>link with all other Windows 95 users on the block.
>
>> > Scalability. NT has a "scalability wall" of about
>> > 100 concurrent users per machine.
>>
>> Funny, I'm posting this for an NT USENET server
>> which takes a 38K group feed and regualrly supports
>> 250-300 simultaneous users, and that is fairly slimly
>> confgurted (by today's standards) Pentium Pro machine.
>
>Sounds like you're due for a CALS audit - can't wait to see you get
>that $15,000 bill from Billy. If your company is public, maybe he'll
>take it in equity. What's 1% of your company worth - about $1000?

You need to read the licence, you don't need a CAL for a IP like connections such as NNTP.

>
>It's not hard to put together a newsgroup server on Linux that runs on a
>486/100.

I'd like to see you run a full feed (35K+groups, 2GB & 1.5-2million articles inbound/day) on a 486 with any OS. If you seriously think that's possible then you've obviously never tried to run a full newsfeed. The machine in question took over from a Digital Alpha server running DEC UNIX about two years ago at which time the feed was roughly half what it is today, and the DEC box was on its knees.

> Are you also serving web pages? (how many pages/second). Are
>you also serving ASP pages? (how many rows/second). Are you taking
>updates? (how many inserts/second).

Newservers whether UNIX or NT are usually dedicated to the task, this machine is no exception.

>
>> > Linux has shown itself to be extremely scalable. Linux can run
>> > effectively on an 80386 machine with as little as 8 meg and a 20
>> > meg hard drive (using network support). It can be scaled up to
>> > Alpha, UltraSparc, or PPC G3 chips with a gigabyte of RAM and both
>> > RAID in software and RAID in hardware, including multiple SCSI and
>> > network cards (ethernet or ATM).
>> >
>> > Linux also supports SMP systems of up to 16 processors and can
>> > run number
>>
>> If you are going to say that LINUX scales to 16 processors,
>> you are going to have tpo point us at some benchmarks that
>> demonstrate this, I think you'll
>> find that hard to do. By the same metric (i.e what the kernel can
>> theoretcially support) NT can handle 32 processors.
>
>Actually, Linux followed the UNIX lead and focused on SP and MPP
>technologies such as BEOWulf, ACE/TAO, and CORBA.

You are moving the goal posts, if you want ot argue that LINUX currently offers some very attractive loosely coupled cluster capabilities, I wouldn't argue. But we both know that that has no bearing on the claim that LINUX SMP scales well on a 16 CPU SMP box.

> Actually,
>Linux 2.0 did NOT scale well in SMP environments due to it's single
>process table and single spinlock. More advanced SMP configurations
>required the 2.1 kernel and special recompiles. Even 2.0 requires
>tuning since the number of spinlocks needs to be adjusted based on the
>number of processors (having 8 spinlocks and tables on a monoprocessor
>system would be wasteful - having 2 spinlocks on a 16 processor system
>would be messy as well. Solaris, AIX, and HP_UX have all gone SP
>and/or MPP.

So, no defence of the claim that LINUX scales well on 16 CPU SMP boxes. To say that Solaris, AIX and HPUX have gone MPP is disingenous, all of them have demonstrated capabilities to scale well on 16 CPU SMP. LINUX has yet to demonstrate scalablity on 2-way SMP which puts it behind NT in this respect, since NT has demonstrated excellent scalability on 4-way and quite useable on 8-way.

>DEC (now COMPAC)has always used clustering for
>scalability. Microsoft offers WolfPack but the standby system is
>passive.

Microsoft has never claimed scalability for MSCS. Though with the IP Load Balancing stuff they are now bundling they can do pretty well on web type applications.

>There are third party cluster packages (from HP), and there
>are MP packages such as MQ, but these all boil down to the
>per-processor capabilities of the base machines. Since SMP machines
>are substantially more expensive than single processor machines, and
>since there's almost no limit to the number of processors in SP/MPP
>systems, it's no big surprise that Linux will scale more cheaply than
>NT.

Depends on the application, if the application lends itself to a Beowulf like approach, say rendering images, then yes LINUX rocks, but trying building a large scale database using Beowulf technologies.

--
Nik Simpson