Re: Max Size Datafile in 10g

Mark Townsend <markbtownsend_at_comcast.net> wrote in message news:<6SP1d.51449$D%.11110_at_attbi_s51>...
> Howard J. Rogers wrote:
> > Mark Townsend wrote:
> >
> >
> >>Joel Garry wrote:
> >>
> >>>"Howard J. Rogers" <hjr_at_dizwell.com> wrote in message
> >>>news:<4146fc78$0$5727$afc38c87_at_news.optusnet.com.au>...
> >>>
> >>>
>
> >>>>
> >>>>If I thought for a minute that anybody on the face of the planet, apart
>
> >>>>from the CIA, actually *needed* 8 Exabytes, I'd care enough to look it
>
> >>>>up...
> >
>
> > He referred to some research done at UCLA on the total information
> > quantity in the world currently at 1 Exabyte, but I don't recall who
> > did it.
>
> >
> >
> >>Two potential areas that I know of (at least, we cite them as the reason
> >>we lifted the restrictions in 10g) that may cause data explosions in the
> >>next 5 years
> >>
> >>Life sciences (storage of 'individualized' genetic code maps ?) and CERN
> >>(the new linear accelerator and the hunt for the God particle)
> >
> >
> >
> > Really?
>
> Really.
>
> > Let me do some maths for a second (warning! warning!).
> >
> > The human genome is about 3,000,000,000 bases long.
> > Each base is just a single (English) letter, so can be represented in 1
> > byte. A human genome would thus require 375,000,000 bytes of storage, which
> > is about 375MB.
> >
> > For the approximately 6 billion people on the planet, you would therefore
> > need 375MB*6,000,000,000 = 2,250,000,000,000MB, which is 2,250,000,000GB,
> > 2,250,000TB, 2,250 Petabytes or 2.25 Exabytes.
>
> Right - and if the 1 Exabyte figure for all current information is
> correct then this would represent a 200+% increase in a very short time
> frame. Sort of what I would call a data explosion.
>
> > So you could comfortably fit an individualised genetic code map for every
> > person on the face of the Earth in less than one third of a single bigfile
> > tablespace in a single database.
> >
> > Were Oracle corporation anticipating some sort of population explosion in
> > the next five years, or something?!
>
> Nope - obviously there is a fair amount of serendipity in the 8 Exabyte
> figure. I don't think we seriously consider that anybody will need this
> in the short term future. However, it was fairly easy to predict that
> petabyte plus storage environments will be common place in the next 2-3
> years, especially as more and more digital data becomes managed online
> for longer periods of time (Oracle itself already has around 3 petabytes
> of storage inhouse - including (and I joke not) over 3 terabytes of
> predominantly powerpoint content in a single database)

Listen to Scott McNealy. Increase productivity by prohibiting ppt's. :)

>
> So we knew limits needed to be raised, it just so happens that the
> bigfile work gave us a nice high 8 exabyte watemark. And we had a couple
> of customers that would validate out thinking for us, and can see
> themselves going to multiple petabytes in the next few years.
>
>
> > Mathematical joking aside, it's an interesting idea: two totally
> > off-the-planet requirements dictating the internal workings of a
> > mass-produced RDBMS. I dare say it doesn't happen very often.
>
>
> You may be surprised - there are a number of them.
>
> Cern, for instance, will generate around 10 Petabytes of info a year
> when they fire the accelerator up. It could run for 10-20 years.
>
> A certain company that does record and backup management and electronic
> vaulting (basically for the world) predicts massive movement of content
> from paper (and glass) and from current tape and offline storage to
> online-all-the-time storage over the next few years - and these guys
> literally have mountains (hint) of content that they manage - films,
> xrays and medical records, 50 years of email, even source code :-).
> Digitizing and storing everything that Hollywood (and Bollywood) has
> produced will require huge storage.
>
> Another group (not a company) wants to maintain the family tree history
> of every person that has lived on the face of the earth to date. Combine
> this with medical history and genomic maps and the figure is very large.

Just tell them it's not OK to convert dead people! They agreed not to do that!

>
> The real kicker - sensor data - imagine a world where EVERYTHING is
> tagged with RFIDs (including, perhaps, even money), and sensors

"...put a $50 bill in the microwave and Hamilton's eye exploded..."

> everywhere detect what is currently in and out of range, going from and
> coming to. Continuously. Or every vehicle on the face of the earth
> reporting current engine statistics and mileage per gallon information.

{For those outside the US,] there is a criminal trial going on (in Redwood City, no less), where the police put a GPS tracking device on the truck of the fellow suspected of murdering his pregnant wife. It has come out in trial that at one point the GPS recorded his Chevy was going 30,000MPH...

> Or every household reporting electricity usage at the 5 second interval
> for spot buying from producers/suppliers. Or in the note too distant

When they put in my new water meter, nobody told the meter reader about it (even though the new one is obvious to casual passers-by, and they removed the old one). So they've been "estimating" my usage based on past usage, which is way under, because the new meter has higher pressure, so more water gets used on landscaping while I recalibrate it. So my next water bill will be several thousand dollars...

> future - the human ADDM device that collects and reports on our own
> individual health statistics.

Don't get me started on ^*%^*!#@ programs that keep kicking me out as non-average! I'm not dead yet! I have a ridiculously low Medical Record Number at Kaiser, that should count for something...

Seems that technology is outpacing analysis...

>
> And then you have the mapping guys - earth maps, sea maps, space maps,
> body maps, genomic maps, mind maps. Maps with multiple dimensions. Maps
> with temperatures, frequencies etc etc etc. The sky is literally the
> limit for this.

And all I want is a stable link to "my house from space." http://terraserver.microsoft.com/addressimage.aspx?t=1&s=10&alon=-117.21453416&alat=33.22836488&w=1&ref=A&Lon=-117.2142639974&Lat=33.228288466

>
> And then this is just the raw stuff - to use all this data, it has to be
> indexed and summarized and backed up - which could increase the actual
> storage foot print by a factors of 10 or more.
>
> And the thing is is that this is just what we can see now. Storage
> prices will continue to plummet, and new applications will be developed
> to take advantage of this fact - the iPod for instance. We don't even
> know what the devices of the future will require.

You know what else is plummeting? Use of error correction. Bad data!  Bad! Here comes the rolled up reuseable newspaper!

>
> >
> > You *sure* it wasn't just so the marketing department could say theirs was
> > bigger than IBM's??!
>
> Nope - we will see single figure petabyte databases very soon (they are
> being built as we speak), and within 10 years we will see 100+ petabyte
> databases and some early exabyte environments. And it's not just
> marketing - Jim Gray has an interesting presentation on this - see
> http://www.research.microsoft.com/~Gray/talks/Gray%20IIST%20Personal%20Petabyte%20Enterprise%20Exabyte.ppt

Ahh, thank you for that. Don't blame Rich, my memory was off by a factor of 10 and the wrong UC campus:
http://www.sims.berkeley.edu/research/projects/how-much-info-2003/execsum.htm#summary

jg

--
@home.com is bogus.
"A libertarian is a conservative who still gets high." - Drew Carey