Re: RE : Recovery in a Replicated environment

Thank you all for your responses to this question.

• Stephane Faroult <sfaroult_at_oriolecorp.com> wrote:
  > Tina,
  >
  > One of my customers uses a home-made, near
  > real-time master-to-master
  > replication for security trading (just to say that
  > when somebody goes
  > wrong there is big money at stake). I have always
  > find recovery in this
  > type of configuration to be the hardest part. In
  > case of crash, you
  > switch to a replicated database which, by
  > definition, is (more or less)
  > up-to-date. Fine. Now the problem is that after you
  > have repaired the
  > crashed database, it is no longer in phase with the
  > others and
  > 'back-to-normal' is very difficult. In the initial
  > design for my
  > customer's replication, there used to be a
  > transaction log table which
  > had to be exported, re-imported into the recovered
  > database and
  > re-applied. There is now on this database an
  > activity which is much much
  > higher than what it was when it was designed, and
  > the tests we have
  > carried out have shown that the official recovery
  > procedure was indeed
  > (much) slower than rebuilding the database from
  > scratch - just what we
  > do when we add a new node to the network. As a
  > result, for recovery what
  > we do is that we wait until markets are closed at
  > the site we want to
  > restore plus the one we want to clone (increasingly
  > difficult these days
  > - there are still W/E, fortunately), download the
  > really useful data
  > (i.e. we forget about audit tables and the like
  > which are not essential
  > to business needs), 'tar' and 'gzip' it, transfer
  > the file and reload
  > everything at the other end with SQL*Loader and the
  > direct path (the
  > best way to minimize both file size - we have sites
  > in America, Asia and
  > Europe, it may have to travel a long way - and
  > reloading times). We are
  > lucky to have reasonably sized databases (a few
  > Gigs) and if the whole
  > system operates in 24x5 mode this is not true at the
  > local level. An
  > interesting side-effect of this strategy is that the
  > most important
  > sites for the business have a backup machine which
  > is replicated as
  > anything else, and that there is no other backup -
  > in the worst case, we
  > can get the data from another continent.
  > We also have a tool for resynchronizing 'live'
  > data if need be, and
  > well, all this has been in operation for three years
  > now and we have
  > survived the premature death of disk controllers in
  > New-York and Zurich.
  >
  > --
  > HTH,
  >
  > Stephane Faroult
  > email: sfaroult_at_oriolecorp.com
  > Oriole Corporation
  > Voice: +44 (0) 7050-696-269
  > Fax: +44 (0) 7050-696-449
  > Performance Tools & Free Scripts
  >

  ---

  
  > http://www.oriolecorp.com, designed by Oracle DBAs
  > for Oracle DBAs
  >

  ---

  
  >
  > >
  > > Hi all.
  > >
  > > We are using multi-master replication using Oracle
  > > 8.1.6 running on Solaris 2.6. Although we are set
  > up
  > > for multi-master replication, there will only be
  > > transaction activity at one site unless the
  > primary
  > > site is lost, in which case we'll be switching to
  > the
  > > secondary site.
  > >
  > > Does anyone have any experience performing
  > recovery in
  > > this type of environment or know of any papers
  > which
  > > outline what steps should be taken for performing
  > > recovery in a replicated environment using
  > different
  > > recovery scenarios.
  > >
  > > I would appreciate any help I can get in this area
  > as
  > > I have been having trouble finding any sources
  > which
  > > address this with any level of detail.
  > >
  > > TIA,
  > >
  > > Tina
  

  ---