Re: implementing a database log

Christoph Rupp wrote:
> Hi,
>
> i'm the author of a simple open-source database engine. i'm currently
> adding recovery/logging, and i have some practical questions about the
> implementation.
>
> I have 3 options.
>
> 1. a physical log based on modified pages - whenever a page is
> modified, a before/after image is stored in the log.
> pro: easy to implement and to test
> con: logfile will become huge!
>
> 2. a physical log based on modified bytes - whenever a database key or
> record is modified, only the modified bytes in the file are logged.
> pro: logfile is slim, very fine-grained control
> con: tedious to implement, error prone, difficult to test
>
> 3. a logical logging - information about the high-level operation is
> stored in the logfile. honestly, i don't know how i could implement
> this, since one high-level operation is often splitted into several
> physical operations, and if one of them fails i am not sure about the
> consequences...
>
> currently, i'm going with 2), but as i wrote above, it's hard to
> implement and i'm not happy with the way i'm going.
>
> what would you recommend?
>
> thanks for your opinions,
> chris

Several advantages of logical logging, not the only ones, I'm sure:

1. Likely more insensitive to future changes in physical organization/operations.
2. Potential to reconstruct old db values with additional constraints applied (I don't know how many times this would have avoided much more awkward surprise requirements).
3. Ability to shadow via another physical db, eg., for read-only audit or various performance requirements (my favourite advantage, maybe most people would favour #1.)