Re: interMedia Text Index Rebuilds

Jeremy Ovenden <newspostings_at_hazelweb.co.uk> wrote in news:MPG.18b71b0ed09481bc9896bb_at_news.cis.dfn.de:

> I believe that search performance gradually degrades and that periodic
> complete index rebuilds are recommended.
>
> Am I right so far?

Yes you are ... the performance degrades because the indexes fragment. This is a function of how close to real-time you're indexing your documents. So if you can delay the indexing by a minute or two, you minimize how often you'll need to rebuild your indexes.

> The problem I have is that no new documents can be added whilst the
> index rebuild is in progress. Is there an elegant way to achieve re-
> indexing whilst still being able to add documents?

You have to bring the portion of the system that requires indexing offline. What we ended up doing was writing our own parser so that the rebuild would run considerably faster.

The stock parser is great if you are dealing with many different document types. In our case, we only needed to parse text within HTML tags. The documentation was (actually!) helpful on how to build your own parser.

-- 
Pablo Sanchez, High-Performance Database Engineering
http://www.hpdbe.com