Re: Design needed: large arrays

> I have to handle large arrays of date from measurements in Oracle.
> These are of about 1000x1000 points, but can be 400x1000 or 10000x100
> as well. These values have to be written, read and deleted as quick as
> possible.
> I tried several design, but found no solution with appropriate performace.
>
> How would a experienced designer handle this kind of data?

Handling data of this form is a big concern at my office, and I am currently evaluating software to perform this kind of data management. In our case, we have many thousands of time series (one-dimensional arrays) and also many multi-dimensional arrays. We are looking at two approaches: (1) using software with array-handling that is layered on top of a relational database (Oracle) and (2) storing arrays in an extended RDBMS (InterBase from Borland, which has arrays of up to 16 dimensions). Performance with the former approach will a factor.

I think you have two options: (1) replacing Oracle with either InterBase or an Object DBMS, and (2) using BLOBs in Oracle or some other RDBMS to store your arrays, in which case you'll need to write indexing functions and functions to turn the BLOB data into numerical datatypes. The problem is that a conventional RDBMS like Oracle simply isn't designed well for storing array data in conventional table structures. You need some kind of object extensions, either BLOBs or (even better) a native array datatype like InterBase's.

                                        Seth

• * * * * * * * * * * * * * * * * * * * * * * * * * * * Seth Grimes, Consultant Directorate for Computers and Communications Organisation for Economic Co-operation and Development OCDE-DCC/SDD phone: (33-1)45.24.19.48 2, rue Andre-Pascal fax: (33-1)45.24.19.51 75775 Paris, France Internet: grimes_at_oecd.fr