Re: What constitues a VLDB?

From: Mark Rosenbaum <mjr_at_netcom.com>
Date: 1997/04/03
Message-ID: <mjrE82Cq9.2GK@netcom.com>#1/1

In article <3342ABED.6A93_at_dshs.wa.gov>,
James F. Valenti <valenjf_at_dshs.wa.gov> wrote:
>Hi!
>
>DBA's have to do extra work when planning and configuring a VLDB.
>What, in your opinion, establishes a database as being Very Large these
>days?
>Is it size alone? Or is it a combination of size, level of concurrency,
>transaction rate?
>If it's size alone, where is the threshold? 10 GB, 100GB, TB's?
>
>

James,

I've been doing parallel processing (SMP & MPP) for a dozen years now and it seems that every 3 to 4 years everything improves by a factor of 4.

Processors increase in performance by 4 Memory increases in size by a factor of 4 Disk price decreases by a factor of 4

Setting disk size or MIPS rating has not been useful when a single unit increases by a factor of 100 over a 12 year time period.

OTOH the number of disk and processors has only increased by a factor of 2 to 4 over the same 12 year period on high end machines.

Currently over a dozen processors and over 100 drives is begining to push the limit. At this size you really do need to change the way things are designed to accomendate the size of the system.

Operationally you cannot backup the entire disk farm in one night and have time for much else so the data should be partitioned into static and dynamic areas. Backup the static areas twice (yes 2 times) and the dynamic areas every night. RAID can help here.

Operationally you will also want to configure the system so a single processor board or memory board can fail and the system will be able to reboot in a deminished capacity.

From a performance perspective you need to spread the data over many drives to avoid bottlenecks (RAID can also help here).

Also memory and disk are getting large enought now that 64 bit OSes are needed to take full advantage of large systems.

Hope this helps

Mark Rosenbaum			Otey-Rosenbaum & Frazier, Inc.
mjr_at_netcom.com			Consultants in High Performance and
(303) 727-7956			Scalable Computing and Applications
POB 1397			ftp://ftp.netcom.com/pub/mj/mjr/resume/

Boulder CO 80306 Received on Thu Apr 03 1997 - 00:00:00 CST