Re: 21 terabytes at NYNEX

In article <318E022C.30A1_at_datamgmt.com>, David M Walker <davidw_at_datamgmt.com> wrote:
>Building a system to hold 21Tb has its problems. These are not
>really related to the file system size of 2Gb as the database
>would have to be raw anyway.

Raw partitions typically are limited to the same 2 GB limit as cooked. It's not just the partition size but also the number of files/partitions that can be open simulaneously (typically 1022) that is the problem.

>If we assume 10Gb per disk, 12 disks per SCSI channel, 4 channels
>per Disk controller card you would need 44 disk controllers.

12 disks per SCSI is quite a few. Typically 4 drives can saturate a SCSI 2 controller. If the data is carefully layed out you may be able to get up to 12 drives on a controller without too much contention.

The SGI challenge has (8*6) or 48 controllers the Cray 6400 superserver (Sparc based SMP) has 64 (I beleive)

There seems to also be support for HiPPI connected disk. This may reduce the controller contention since HiPPI runs at up to 100 MB for 32 bit HiPPI.

>In practice most SMP platforms support about 16 mid plane slots
>(Though look at some of the new offerings from Sequent and others)
>
>If we need 8 slots for processors and 2 slots for memory then we
>are left with six slots for controllers. This gives us about 2.9Tb
>of raw disk to start with. In practice, this would reduce down to
>2Tb after RAID, O/S, Swap and load areas where taken into consideration.

The problem is not hooking up that many drives to one system, the problem is finding any software (RDBMS) that supports that much disk. I believe that both Oracle and Informix both have 64 bit version that currently runs on DEC Alpha and will soon run on SGI (Irix 6.2 or later). I am scheduled to be working with both of these RDBMS next month. If there is interest I will be happy to post my experiences.

>21Tb therefore needs to be served by 12 Systems with a distributed
>system, or more likely a using a series of data marts built on
>a transaction repository.

I tend to agree because currently SCSI 2 FWD (Fast, Wide, Differential) will not support the number of drive necessary unless most of the drives are idle.

Mark Rosenbaum				Otey-Rosenbaum & Frazier
mjr_at_netcom.com				Consultants in High Performance and
(303) 727-7956				Scalable Computing and Applications