Re: Making a tree with "millions and millions" of dynamic nodes

Christian Fowler <google_at_NOSPAM.gravesweeper.com> wrote in message news:<6b-dnQJnDI6YvlCiXTWc-g_at_speakeasy.net>...
> I have a VERY LARGE pile of geographic data that I am importing into a database (db of choice is postgres,
> though may hop to oracle if necessary). The data is strictly hierarchical - each node has one, and only one
> parent. The depth should not exceed 6 or 7 levels. The initial import will have about 6 million leaves, and
> 3 million branches.
> For selection, it is crucial for me to get:
>
> 1. path generation speed
> 2. immediate sibling speed
> 3. children count speed
>

I don't know how high your requirement for having an SQL interface is. At my company we wrote a real-time database for the local number portability, with over 40M entries, with support for variable length phone numbers ( up to 10 digits ). The customer wanted the longest phone number match possible. example there could 1, 12, 123 in the database and the number 1234 would match to 123.

We did it with C++ and a memory mapped file for persistence. We had to use a 64bit cpu and 12G of RAM.

we can do over 100K insert/search/delete's per second. a relatively flat or equal performance for each operation ( depends on number length ).

If real-time performance is that important, then you might have to customize. Received on Wed Dec 10 2003 - 23:23:32 CET