Re: Relational Model and Search Engines?

After a long battle with technology, Leandro Guimarães Faria Corsetti Dutra <leandro_at_dutra.fastmail.fm>, an earthling, wrote:
> Just realised it's become offtopic... sorry...
>
> Em Mon, 03 May 2004 22:23:56 +0000, Nick Landsberg escreveu:
>
>> In-memory access does not have to go across any bus to any
>> device. When you are dealing with 8,000 TPS this does have an
>> effect.
>
> So does caching.

Ah, but the typical "cached pages" approach does differ in that there needs to be:

1. Code that evaluates whether or not a page is in memory;
2. Code that pulls in data if it's not;
3. Code that pushes data out to disk.

That code disappears, as does the logic that it would have executed.

There's some very similar parallel theory that OSes like Linux and FreeBSD might be made faster if you eliminated the capability to page to disk, and required that there be enough RAM to run all processes in memory. It is claimed that QNX is aided in its performance by this precise absence of page-swapping code.

>> Commits are done to the disk, as usual (to the journal
>> file). Depending on the application, these may be either synchronous
>> (right now), or deferred until the next millisecond or the next 10
>> commits or something like that. If you can live with losing the
>> last 10 transactions in case of a crash, you can live with an
>> in-memory database. (It's a tradeoff.)
>
> So, it is basically for throwaway data. I was hoping for
> something more interesting.

If you want a robust transaction log, that will have to get serialized to disk on a steady basis; that is presumably what Cach\'{e}, TimesTen, and such do. Having to serialize this upon COMMIT will certainly lessen performance, but all of the virtual page-management code disappeared, so performance should still be pretty good.

-- 
(format nil "~S_at_~S" "cbbrowne" "acm.org")
http://www.ntlug.org/~cbbrowne/sap.html
God is a comedian, playing to an audience too afraid to laugh.
-- Voltaire