Re: Stock market tick data

From: Michael Peppler <mpeppler_at_peppler.org>
Date: Mon, 04 Jun 2001 11:26:17 -0700
Message-ID: <thnknojkuup58f@corp.supernews.com>

In article <3b166b8f$0$18891$272ea4a1_at_news.execpc.com>, "Paul Tomko" <tomko_at_earth.execpc.com> wrote:

> In article <9f4rt1$350$1_at_rex.ip-plus.net>, Ian Ledzion
> <ian.ledzion_at_lgxbow.com> wrote:

>>What do you mean by tick data?

>
> Every time a transaction occurs on a particular instrument, I want to
> record the time, number of shares transacted, and various other
> information. The thing that worries me is that even one stock can have
> 10s of thousands of transactions in one day, and if you are tracking
> hundreds of stocks, then I could anticipate that loading them into a
> RDBMS in real time could be a real chore. The sheer amount of data
> accumulated over time would lead to huge tables as well. I could see a
> need to have at least one table for each stock, although each table
> would have the exact same record layout. Further, I could see a possible
> need to break down each stock table by a time period.
> Has anyone had experience in recording real time stock data in an RDBMS
> and could share some insight?

I've done some of that, though not recently.

While you may want to partition your stock tick table, I would strongly recommend that you have one logical table with all the data, and as much historical data as you want to keep (remember that there are 10s of thousands of instruments that you may want to keep data for!)

The major issue here of course is the volume, both of inserts during the day and of raw storage after a few days... However, the inserts shouldn't really be a problem unless you have a need to execute a lot of queries on the same data at the same time - I had a system where we were storing real-time tick data for the commodity markets using a Sun SS-10 with a paltry 128 mb RAM with no problems...

Michael

> Thanks,
> Paul
>

>>"Paul Tomko" <tomko_at_earth.execpc.com> wrote in message
>>news:3b15169e$0$18895$272ea4a1_at_news.execpc.com...
>>> Has anyone worked with storing tick data in an RDBMS? I would be

interested

>>> in hearing some examples of how this has been implemented. If there
>>> are any Case studies, books, or data models out there, I would like to
>>> hear about them. I have some ideas of my own on the subject and would
>>> like to dialog with people who have done an implementation. Thanks,
>>> Paul Tomko
>>> --
>>> Paul Tomko     paul@tomkoinc.com    http://www.tomkoinc.com 10000+
>>> Humorous Quotes              http://www.tomkoinc.com/quotes.html
>>> "Behold the warranty ... the bold print giveth and the fine print
>>> taketh away." - fortune file
>>
>>
>>

-- 
Michael Peppler - Data Migrations Inc. - mpeppler_at_peppler.org
http://www.mbay.net/~mpeppler - mpeppler@mbay.net
International Sybase User Group - http://www.isug.com
Sybase on Linux mailing list: ase-linux-list_at_isug.com

Received on Mon Jun 04 2001 - 13:26:17 CDT