Re: Performance and reliability of c-tree vs. Sybase or Oracle

Josef Holzer wrote:
>
> I am currently using a c-tree (Faircom) database for collecting
> pricedata of fiancial instruments. As I want to improve the reliability
> of the database as well as to increase the number of datarecords, I
> would like to move to a more robust database like Sybase or Oracle. But
> I do not want to lose performance compared to my existing c-tree
> database that also runs fine on a desktop computer.
>
> The database should be able to store about 50.000 instruments that have
> updates between 1 and 10.000 records per day (some instruments even more
> often). At some peak times the database has to handle about 1000 updates
> per second.
> To handle such update frequencies I have implemented some caching as
> well as buffering: I store blocks of small price records (Instrument,
> Time, Last, Vol) in one database record.

I have done similar things using a Sybase back-end, and using a Sun Sparc 10 as the server. We were storing a live data feed of futures & options data, and it worked pretty well, although we sometimes had a pretty large backlog when the volume got really large (such as when the US labor report is issued on the first Friday of every month).

We had a dynamically allocated buffer on the incoming feed that could grow to several megabytes in worst case scenarios, and we stored tick-by-tick price information in flat files as we usually loaded the entire files when we needed them. We optimized the number of records in the historical database by storing data minute by minute in the following format:

time open high low close [p2, p3, p4...]

where high, low, close and p2, etc. are stored as differences to open (the first price in the minute).

We had to write some specialized software to manage the flat files, but all in all it was a pretty fast and robust system.

Michael

-- 
Michael Peppler    -||-  Data Migrations Inc.
mpeppler_at_mbay.net  -||-  http://www.mbay.net/~mpeppler