Re: Theory of Timeseries extensions to SQL and database

Hi Bob,

I do understand the differences between the physical representation and the logical model. I have already built a relational database from scratch for the PenPoint OS in the 92 with mainly CJ Dates book as my source for knowledge. I just didn't express myself very well in the posting.

I understand why kx systems pollute the logical model with arrays in kdb. Despite what they say about their database it is very specialised use. They have eliminated the need for locking, a big overhead, by making the database single process, so queries have to queue. The target users (and the creators of kdb) I believe are very familiar with using vectors and that is why they are exposed as first class objects. The language k, used for the functional extensions for operation on the timeseries, is conceptually based on APL, it is a vector based language.

My interest comes from having a product that manages market data feeds, not TIC data but day end pricing, with the potential of 20 years pricing history. I am exploring what my options are if I want to offer my potential clients not only management of the ongoing data feed but also a platform for doing their timeseries analysis. For example, FAME offer over a dozen Numeric scalar functions, over 2 dozen Numeric Timeseries and over a dozen Numeric series functions for clients to operate on the data in the database. So you need exceptional performance when operating on large data sets and as simple an expression of the type of functions clients want to use on the data. The former I think being easier to achieve than the latter. Is it possible to build the latter on top of standard SQL?

Maybe I am not telling you anything you don't already know, but I am testing out my understanding.

Regards

David

"Bob Badour" <bbadour_at_golden.net> wrote in message news:xrjx9.127$nI1.38570054_at_radon.golden.net...
> David,
>
> You need to understand the difference between logical representation
> (relations) and physical storage (hashes, heaps, b+tree indexes, ISAM,
> r-tree, clustering, partitioning, etc.)
>
> A good dbms will allow you to manipulate the data exactly the same (ie.
> using relations) regardless of how it physically stores the data. From
what
> I understand, kdb is actually a good example of how this is possible;
> although, I also understand they make the mistake of polluting the logical
> data model with arrays.
>
> Cheers,
> Bob
>
> "David" <david.wynter_at_btclick.com> wrote in message
> news:apbre3$68e$1_at_venus.btinternet.com...
> > Hi,
> >
> > I did a Google search on timeseries and database and found some research
> > called Sorted Relational Query Language. Also SQL-TS. I have a few
> questions
> > beyond what I found.
> >
> > Does anyone know where there is reseach on the method of storing columns
> > instead of relations, called inversion I believe? What I am after is an
> > understanding of the techniques available to manipulate (i.e. query
> against)
> > result sets as big as a 500,000*5,000 matrix of a 2 element aray
> containing
> > a unique identifier and a numeric value that would perform very fast.
> >
> > Are there any are there any implementations of SRQL or SQL-TS?
> >
> > Thanks
> >
> > David
> >
> >
> >
> >
>
>
Received on Tue Nov 05 2002 - 10:14:42 CET