| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
 |
 | |||
| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
|
| |||
Home -> Community -> Mailing Lists -> Oracle-L -> Re: Disaster Recovery Fallback Sites
>From the "Q" you have, it sounds like someone's marketing people have been
there! EMC doesn't necessarily replicate the entire array. Whether it is
"nearly instantaneous" or not depends on a lot of factors - transaction
load, distance, etc. I would also contend that any talk about using
geographically dispersed SRDF-maintained arrays for "operating system
fail-over" is basically over-generalized marketing propaganda. EMC doesn't
sell clustering solutions, they sell storage solutions! Don't get me wrong,
I have recommended and implemented EMC solutions of the type you describe,
but, depending on your needs, storage replication alone may or may not be an
appropriate solution. For one thing, considering that EMC uses pure
physical disk replication, most corruptions in the primary will be
religiously propagated to the secondary. You might not want this to happen
"nearly instantaneously"!
Without delving into all the tangential availability issues (network, power, application restart, ad infinitum), when considering Oracle availability, there are essentially two areas of concern - instance and database. Consider these two basic classes of failure, their modes of failure, and how to handle them. Consider also that there are preventative and restorative measures for each.
Instance failure is the case where the database itself is intact, but the
instance running against it is inaccessible or down. One common cause
of this sort of failure is hardware failure on a particular system - CPU
failure,
mainboard failure, NIC failure, etc. Also consider things like memory
leaks that might crash the running system or make it unusable.
Preventative measures include hardware redundancy, proper testing of patches (OS, Oracle, application, ...), etc.
The most basic cure is to fix the root problem and then restart the instance. There are essentially three other restorative measures for this sort of failure - all similar in concept. All involve redundancy of the node and the software. In increasing order of complexity, sophistication, and level of availability, they are:
Note that none of the above, by themselves, provide protection against a database failure. They are all solutions where, in the event of an instance/system failure, another instance/system takes over for the failed one - running against the same database.
Database failure is the case where there is corruption or loss of datafiles, redo logs, or control files. The number of modes of failure is greater here - many are quite survivable, some are not. For example, the loss of a single control file should not be a major disaster! The loss of some datafiles, in some systems, may be non-critical in the near term. The loss of a single redo log file may not be critical.
Preventative measures include multiple members per redo log group, multiple copies of the control file, RAID, multiple controllers, and such - all familiar to DBAs.
General classification of restorative solutions is a bit more difficult and the potential solutions are greater in number. At the most basic level is simple backup and recovery. Other, more sophisticated, solutions fall into essentially two classes - physical (disk) replication and logical (transaction) replication. However the distinctions are not always clear and hybrids abound.
Physical data replication involves making, perhaps periodic, physical copies of the database and/or mirroring writes to the database files. Simple backups fall into this category. Some other technologies, such as EMC's Timefinder, BCVs, and SRDF and also fall into this category since they copy the physical structure of the database and changes to it without knowledge of or regard to logical transactions. They simply mirror physical (controller or OS) writes and may require some additional action, such as applying archive log files, before the "replicated" database can reach a consistent and usable state. For example, BCVs are essentially Oracle backups to disk, with greatly enhanced recovery time since the data doesn't need to be completely copied back to the original location before processing can resume. Standby databases, in the most elemental form at least, also fall into this category and are yet another variation on the backup & recovery theme.
Logical replication is different in that the physical structure of the database may be different - the datafiles may be sized differently, they may have different names, etc. Oracle advanced replication is a solution of this type. but there are others. The idea is that you are replicating logical transactions or writes, not physical disk writes. Some log sniffing technologies are essentially this class of solution also, but may have some "physical" constraints (e.g. is ROWID critical?). Another seldom mentioned solution of this type is to use a TP layer mechanism to replicate transactions to a backup system. This can be a very complex, but flexible solution. The "redundant" systems in the TP replication model can even be heterogeneous. In the most complex cases, the primary could be DB2 on a mainframe while the "backup" is Oracle on Unix.
In the data/transaction/database replication options, whether physical or logical, there are also considerations of currency. Should it be synchronous or asynchronous? SRDF, Oracle advanced replication, and other such technologies offer this choice. Careful consideration should be given to this decision. Too many times, I have seen people automatically assume that synchronous is better. Even aside from performance implications, it isn't always so. The question that usually stops them dead in their tracks is "Should corruptions be applied immediately also?" For example, if all your solutions are synchronous, then "drop table ORDERS" is applied synchronously everywhere. Is this desirable? I have seen this kind of thing happen, where one outage exceeded the professed "five 9s" goal - for the next decade! One option, for example, might be to maintain a lag between the primary database and the standby database. By delaying the application of archived redo log files to the standby by some time delta, it might be possible to prevent some types of corruption propagation. An ideal solution using standby databases might be to maintain two standbys - one kept as current as possible and one delayed by some well-considered delta.
Note that some database recovery solutions also cover instance recovery. For example, a standby database could be used as a "standby instance" also in the event of a failure of the primary server or instance. It may, however, not be a appropriate instance failure solution for some systems. Since OPS and HA would not need to perform any of the database recovery operations that a standby database would, they would likely be available much more quickly. Since the standby database might be in a remote location, there might be a large number of other additional tasks that need to be performed to use the standby.
In disaster recovery scenarios, like the catastrophic loss of an entire data center, both modes come into play. HA and OPS typically require the nodes of the cluster to be in close proximity. If you lose the entire data center, the entire cluster would be inaccessible.
The key considerations in either instance or database availability are essentially similar to the classic considerations for backup and recovery:
How much data can you afford to lose?
How much downtime can you afford before recovery?
How much can you afford to spend for a solution?
[The traditional "How much downtime can you afford for backups?" is moot in the context of true high availability. Cold backups are simply not an option for most systems where it is desired.]
The best solution for "high availability" for any system depends on how it is defined in context. What is the probability of an instance failure? What is the probability of a database failure? What is the probability of a complete disaster? How much does each of these cost the company? What are the costs associated with different potential solutions? How much does each save? For some, a single standby database alone may suffice. For others, hybrid solutions are essential. For example, one project I did recently had extremely high availability and performance requirements and a suitably large budget. The designed solution was to use 8i OPS for scalability and instance availability, EMC's Timefinder and BCVs for quick database recovery on the primary and also to enable offloading database backups from the primary systems, and a standby database in a remote data center utilizing SRDF to maintain synchronous copies of the online redo logs and the archives log files - for a "no data loss" standby database. However, the application of the archive logs to the standby was deferred for thirty minutes. Thus the standby could be brought up from any state between "30 minutes before failure time" and "failure time - with no data loss".
Striking a good balance between cost, complexity, and availability is difficult - at best. It isn't rocket science - it is harder! By far the toughest part of the job isn't the technology, it is in getting accurate requirements and sufficient budget. Too often the (supposed) requirements are simply a stern mandate from on high for "No data loss and instantaneous automatic failover - an economical and immediate solution". One of the biggest problems is that CEOs/CIOs and the like rarely have a good grip on even the basics. They read each other's wildly optimistic and often outrageously exaggerated statements claiming 99.999% availability and think that everybody else actually has it (when they don't), so it can't be that difficult.
-Don Granaman
[certifiable Orasaurus]
MY Qs. IN CAPITALS BELOW :-
Storage-Array/OS fail-over :- -------------------------- Storage-array vendors, most notably EMC, have taken the concept ofoperating system fail-over one step further. Noting the drawbacks of clustered hardware in the area of disaster recovery and rolling upgrades, they have taken the basic fail-over mechanism at the operating system level, and replaced the clustered hardware concept with the idea of replicated storage-arrays.
All transactions written to one storage-array would be replicated almost instantaneously to another storage-array.
Qs. IF THE DATABASE CRASHES IN THE PRIMARY STORAGE ARRAY BOX , WILL IT BE COPIED TO THE 2nd REPLICATION ARRAY THEREBY CAUSING IT TOO HOLD A CRASHED COPY OF THE DATABASE TOO ? These two storage-arrays can be geographically separated, though the network between them must be extremely large (i.e. OC3 or higher). The computers, then, are not clustered, and do not require the additional hardware and operating system software to achieve clustering. Products include EMC Symmetrix Data Replication Facility (SDRF), and Oracle Support has white papers describing certification testing performed using SDRF at Oracle.
> -----Original Message-----
> From: VIVEK_SHARMA
> Sent: Sunday, September 02, 2001 3:12 PM
> To: 'gajav_at_yahoo.com'; 'ORACLE-L_at_fatcity.com';
> 'oracledba_at_lazydba.com'
> Subject: Disaster Recovery Fallback Sites
>
>
> Disaster Recovery Fallback Sites :-
> --------------------------------
> What are the Different Solutions/methodologies implemented ?
> Is Standby Database the best Solution ?
> What are the Harware/O.S. Solutions available ?
> Which would be better using Oracle or O.S. Solutions ?
> Any papers , Links , best practices ?
>
-- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: VIVEK_SHARMA INET: VIVEK_SHARMA_at_infy.com Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet access / Mailing Lists -------------------------------------------------------------------- To REMOVE yourself from this mailing list, send an E-Mail message to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Don Granaman INET: granaman_at_home.com Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet access / Mailing Lists -------------------------------------------------------------------- To REMOVE yourself from this mailing list, send an E-Mail message to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).Received on Tue Sep 04 2001 - 11:21:20 CDT
 |
 |