Re: search engines

tbandrow_at_unitedsoftworks.com (tj bandrowsky) wrote in message news:<52e031b9.0202151454.4d9eae5d_at_posting.google.com>...
> It's not the search time that's an issue with most RDBMS, its the time
> it takes
> to load up the table you want to search.

You are talking about loading the table into memory? Right?

> If you built a table of
> 100,000,000 rows in Oracle, each with say a word for a key, plus a
> reference to a file,

"Reference to a file"? Are you talking about pointers to binaries/text files?

> you could probably select all of the document ids
> for a given word in under a second or so. It's that fast.
> Unfortunately, sans bulk loading or something, normally you would get
> to insert around 1000-5000 rows per second into your table, which is
> much too slow for building up a full text index - think word == row.
>
> Some people think that "simpler" databases are automatically faster,
> and that support of transactions makes a database slower. This is
> actually not the case. For either MySQL or Oracle, for example, I
> used to insert about a thousand rows per second.

I'm really talking about searches, not loading.

> That was with a two
> field primary key plus four values for the row. You have to do more
> to get that speed out of Oracle, bound statements are a must,

I don't know what a 'bound' statement is.

> but, you
> can get it.
>
> You could also theoretically write your own dedicated indexing scheme
> for that reason.

This seems more likely. How do you go about writing such an index scheme and how is it implmeneted? Is there documentation/literature on this. I'm using Informix.

Thanks

> A lot of people do. In fact, you'll see dedicated
> full text indeces running in parallel with a traditional RDBMS. SQL
> Server does this, as does Oracle.
>
> bruces_at_real-info.com (Bruce Schreiber) wrote in message news:<f1645a5c.0202141058.64c48de2_at_posting.google.com>...
> > How can search engine search through millions of records in just a few
> > seconds?
Received on Fri Feb 22 2002 - 15:22:51 CET