Re: scientific database application

From: Hugo Korwaser <hugo.korwaser_at_dspace.co.uk>
Date: Sat, 21 Jul 2001 23:29:11 GMT
Message-ID: <3B18BEDA.F2ED65DC_at_dspace.co.uk>


There are a number file formats for scientific data.

CDF, Common Data Format, from NASA
http://nssdc.gsfc.nasa.gov/cdf/cdf_home.html netCDF, network Common Data Form, from Unidata http://www.unidata.ucar.edu/packages/netcdf/ hdf, hierarchical data format, from NCSA http://hdf.ncsa.uiuc.edu/HDF

All of these are used for storing large amounts of data from various sources including CFD analysis. The comments below apply to all of them. I am not sure about restrictions on size. I use hdf5, http://hdf.ncsa.uiuc.edu/HDF.

You may also want to look at CGNS, http://www.cgns.org/, which is a standard for CFD data which includes an API and file format. The file format, ADF, is similar in concept to those listed above and could be used seperately.

guytheweird_at_caramail.com wrote:
>
> Hi all,
>
> I do research in computational fluid dynamics and I'm wondering which
> free database (postgreSQL,MySQL,Interbase, ...) is best suited to store
> my datasets. My requirements are as follow:
> -very large datasets (dozens of gigabytes and even more).

Yes, either in one file or in a set of files.

> -data types : structured (multi dim arrays) or unstructured (blobs?)

Structured data is supported by all. Unstructured data depends on the package, hdf5 allows one to define ones own 'record' type similar to a C struct.

> - no high concurrency levels: few data producers/solver ( (<10) and few
> data consummers/visualisation tools (<10)

No concurrency support inbuilt.

> - security, transaction, recovery facilitities not very relevant

No support

> - fast data access, simple query (mainly storage and retrieval of whole
> data sets)

Yes. Most include the ability to at least extract subsets of data.

> - and the last but not the least, the possiblity to embed the database
> directly in my commercial application to distribute the whole in a
> seamless way

I believe you can, but you will need to check.

Regards

Hugo Received on Sun Jul 22 2001 - 01:29:11 CEST

Original text of this message