RE: Question on Exadata X8 - IO

This is a question about business continuation value and convenience. Determining your optimal answer has a lot to do with the time cost per hour at particular times of day/week/month/year and the likelihood you might need to actually recover.  

Set up as described below you are unlikely to lose any data without a direct disaster strike (and read about Axxana and similar devices if you want to protect for that).  

If you are high redundancy, when you lose a single “plex” you are still mirrored on line. If your host crashes there shouldn’t be any drama restarting and continuing, even if one of the two remaining “plexes” are corrupted by the crash. So the frequency of the drama of actually recovering something and rolling it forward is quite limited. You would need a serious double plex crash to occur within the time to replace via normal maintenance the media (if needed) and “resilver” the new third plex.  

If you are only duplex, when you lose a single “plex” during the time to reassign and resilver the lost “plex” you are subject to having to recover files and roll them forward with respect to contents at least partially on that lost “plex” IF a crash event corrupts the surviving relevant “plex.”  

For triple versus double, you pay more in acreage and actual physical i/o in the storage cells.  

IF you are physical i/o bound or acreage bound, it may be worth trying to quantify whether the (slight) additional risk is worth the increase in the very small probability of a corrupting crash whilst on a single plex.  

I believe that is the most important question for you when deciding.

I don’t know why write back flash cache would be limited for only normal redundancy, but I have not dug that deep.  

mwf  

From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Lok P Sent: Friday, February 12, 2021 9:57 AM
To: Oracle L
Subject: Re: Question on Exadata X8 - IO  

Basically i am seeing many doc stating triple mirroring is recommended with "write back flash cache" and some other stating write back flash cache is not possible without HIGH redundancy/triple mirroring. So there is a difference between these two statements because if we decide to go for NORMAL redundancy to save some space and to have some IO benefit(in terms of not writing one more additional data block copy). But we want to utilize the "write back flash cache" option to get benefits on write IOPS. And in this case if restriction is put in place for "High redundancy" we won't be able to do that. Please Correct me if my understanding is wrong?  

On Fri, Feb 12, 2021 at 12:53 PM Lok P <loknath.73_at_gmail.com> wrote:

Seeing in below doc which state its recommended to go for high redundancy ASM disk group(i.e. triple mirroring) in case we are using write back flash cache as because the data will be first written/stay in flash cache and flushed to the disk later stage and in case of failure it has to be recovered from mirror copy. But i am wondering , is this not possible with double mirroring , will it not survive the data loss in case of failure? Want to understand what is the suggested setup which will give optimal space usage without compromising IOPS and data loss.

https://docs.oracle.com/en/engineered-systems/exadata-database-machine/sagug/exadata-storage-server-software-introduction.html#GUID-E10F7A58-2B07-472D-BF31-28D6D0201D53  

Regards

Lok  

On Fri, Feb 12, 2021 at 10:42 AM Lok P <loknath.73_at_gmail.com> wrote:

Hello Listers, We are moving from exadata X5 to X8 and there are multiple reasons behind it. Few of them are , 1)we are almost going to saturate the existing storage capacity(DB size reaching ~150TB) in current X5. 2)And the current IOPS on X5 is also reaching its max while the system works during its peak load.

We are currently having HIGH redundancy(triple mirroring) for our existing X5 machines for DATA and RECO disk group and DBFS is kept as NORMAL redundancy(double mirroring). Now few folks raised questions on the impact on IOPS and storage space consumption, if we use double mirroring(NORMAl redundancy) vs triple mirroring(High redundancy) in the new X8 machine. I can see the benefit of double mirroring(Normal redundancy) being saved in storage space(around 1/3rd in terms of DATA and REDO copies), but then what is the risk wrt data loss, is it okay in a production system? (Note- We do use ZDLRA backup for taking the DB backup. And for disaster recovery we have active data guard physical standby in place which runs in read only mode).

With regards to IOPS, we are going with default write back flash cache enabled here. Is it correct that with double mirroring we have to write/read into two places VS in triple mirroring we have to do that in three places , so there will also be degradation in IOPS with triple mirroring/High redundancy as compared to double mirroring? if it's true then by what percentage the IOPS degradation will be there? And then if it's okay if we go for double mirroring as that will benefit us wrt IOPS and also saves a good amount of storage space?

Regards

Lok

--
http://www.freelists.org/webpage/oracle-l