Re: Linux betas NT in TPC testing, running Oracle8

Bob Hauck > wrote in message ...
>cbbrowne_at_news.hex.net (Christopher Browne) writes:
>
>> On Wed, 12 May 1999 13:52:10 -0500, nik <ndsimpso_at_ingr.com> wrote:
>
>> > I'd like to see you run a full feed (35K+groups, 2GB & 1.5-2million
>> > articles inbound/day) on a 486 with any OS. If you seriously think
>> > that's possible then you've obviously never tried to run a full
>> > newsfeed.
>
>> I suggest that you bounce an email to <sdenny_at_hex.net>. He used to do
>> something less dissimilar to this than you'd think.
>
>What kills you with newsfeeds isn't the feed itself. It is the history
>file, downstream feeds, and readers.

Precisely. If you are receiving 1.5 million articles a day then you are also expiring that many (or the disk space rapidly fills up.) So you are talking about:

1.5Million new files
1.5 Million deleted files
3million history updates
1.5 Million updates to the active file
etc.

The box in question also provides a sanitized feed (no binaries etc) to a second server, so that's another big chunk of I/O and networking. It handles average reader connections of around 250-300 and in addition the box runs various SPAM killing algorithms so it has to applies those algorithms to all the inbound articles etc. Even so, the box is not more than 60% average CPU load, the killer is the IO. The days when you could cobble together a newserver that can handle a really full feed on a 486 box are long gone and it doesn't matter what the OS is.

This stuff is also a great way to stress filesystem code, the application uncovered serious bugs in the journalling filesystem on DEC UNIX when it ran on that box and stressed NTFS to a degree that few normal applications do.

--
Nik Simpson