Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Usenet -> c.d.o.misc -> Re: Oracle performance in high-volume data storage.

Re: Oracle performance in high-volume data storage.

From: Wayne Balmer <wbalmer_at_wainwrights.com>
Date: 1997/04/11
Message-ID: <5ilc3q$jd8$1@picasso.op.net>#1/1

Barry Schader <barry.schader_at_medtronic.com> wrote:

>Petri J. Riipinen wrote:
>>
>> Hi there,
>>
>> We have a data collection running 24h (several processes in different
>> workstations) and we get about 200M of data each day. At the moment we store
>> this into proprietary file for size and performance reasons. Unfortunately,
>> this makes the later analysis of the data a bit difficult with any standard
>> tools.
>>
>> However, now we would like to make the accessing and analyzing of the data
>> easier. That would mean that we would store the data into Oracle (that's what
>> we use elsewhere). The database would be probably running on Sun or HP-UX.
>>
>> Our requirements:
>> - Constant database writing, rate being about 2,5k / second
>> this writing must not cause any blocking to the writer processes.
>> - Reading at the same time in bursts (when someone runs the analyzing tool).
>> - The data will be stored for about 2 weeks and then the data will
>> be deleted every day. So the database will grow up to 3GB or so.
>> - The data contains a timestamp and about 20 numeric fields.
>> - The database is mission-critical and it must be on-line 24h.
>> - The queries will be made on the timestamp + a numeric field or two.
>>
>> I have the feeling that Oracle should handle this without any problems.
>>
>> What is your "gut" feeling about this? Should we look into parallel processing
>> version? Any references to any high-volume storage applications?
>>
>> - Petri
 

>Just a caution:
 

>Be careful about running large queries on the same machine as
>your transaction-processing stuff. I've got a 3GB Oracle 7.2
>database running on NT 3.51 on a 233MHz Alpha box and the CPU
>routinely gets driven to 100% by a single query. This surprised
>me, and it may not happen to you, but it's something to consider.
 

>You might want to replicate your data onto another machine for
>querying (a 'data warehouse'?). You can tune each database for
>its intended purpose, and you can keep data for a longer time on
>your query-only database.
 

>Barry Schader
>Medtronic Micro-Rel

Before I read Barry's reply above I was already thinking. Have one machine for data collection and storage and another machine for analysis. Hopefully, you can take that as a confirmation that it might be a good idea. You need to separate out the business functions you want to accomplish here. I don't know that you really want replication for this, but what you want is proven processes (no manual intervention) to keep the flow of data reliable on the input machine.

You want to automate steps for loading, cleaning, and archiving. After this is running you can analye what time slots or situations might be available for data extraction. Then you want to use proven, canned extraction and summarization processes to put the data into a usable form on the analysis machine.

Wayne Balmer Received on Fri Apr 11 1997 - 00:00:00 CDT

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US