Re: SSDs and LUNs

That's solution dependent, so I can't give a general answer and will require conversations with the storage, O/S and virtualization teams assuming you are organized that way. In addition, you've got external consultants to talk to as well. When talking to them it's important to understand that multiple paths to storage offer two advantages. You want to ensure that the offered solution covers both cases.

1. If one of the paths fails due to a hardware failure, then the system can continue - this is why physical servers typically have 2 HBAs for example.
2. Each O/S device typically can handle X *concurrent* requests (not the same thing as IOPS). X is solution dependent but can be as low as 32. Adding paths adds concurrent I/O request capacity. The term for this is Queue Depth.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/index.html#MPIO_description is a good *simple* overview of how this works in a traditional world, essentially the more HBAs, switch ports and controllers you have the greater the available number of routes to the SAN. Your O/S and SAN people should be familiar with this.

You said this was a VM solution, however. There are a greater number of storage options there and a number of ways that path redundancy and outstanding I/O capacity can be provided. For example disks can be attached directly to the VM just as they would be to a physical host, storage can be provided by the virtualization solution (in a number of forms) and storage might be provided over the network via iSCSI. I've added the list back in, in case anyone has a simple overview of good configuration principles for Oracle storage in a VM world. Again, actual details will be technology specific but the two principles above should allow you and your system administrators to ask good questions of the proposed solution.

Separately you asked how many paths we used. Off the top of my head, the answer is 4, but it could be 8. My advice would be to start small but at least 2 to provide redundancy, you can monitor actual queue depth statistics on Linux using iostat -x. avgqu-sz would be the column of interest. You'd need to compare that to the device queue size your setup is configured for, if the two figures are closed and your I/O is slower than you expect but the storage array says all is fine you've probably got an issue. That said, in most cases, you'll probably find that 2 paths provide enough capacity as well as enough redundancy.

On Sat, Oct 21, 2017 at 8:27 AM, Ram Raman <veeeraman_at_gmail.com> wrote:

> Thanks Niall.
>
> "What you would normally do is to increase the number of paths to the
> storage" How would you do that?
>
> On Thu, Oct 19, 2017 at 2:58 PM, Niall Litchfield <
> niall.litchfield_at_gmail.com> wrote:
>
>> What you would normally do is to increase the number of paths to the
>> storage, and use the mpath device as your ASM disk. You'll get an O/S queue
>> per path that way. It is, of course, possible to generate queueing within
>> the array by being over enthusiastic.
>>
>> As a point of reference, we are happy with larger LUNS (4tb) and more
>> paths for our all flash array based databases. If you do use LUN sizes
>> larger than 2T *and* use ACFS make sure you are on a current version ( >
>> 12.1.0.2.5) otherwise once your ACFS filesystem gets more than 2T of data
>> added to it, you'll lose it all :( https://support.oracle.com/
>> epmos/faces/DocumentDisplay?id=2065748.1
>>
>>
>>
>> On Thu, Oct 19, 2017 at 8:09 PM, Ram Raman <veeeraman_at_gmail.com> wrote:
>>
>>>
>>> "If there is a layer at which you have a single queue to each LUN you
>>> will have an I/O bottleneck" is exactly what I am thinking of. just 2
>>> queues for TB of data?
>>>
>>> Let me read up on the link Connor emailed.
>>>
>>> Rich J: I cannot understand your question since those terms sound new to
>>> me. I will have to research that.
>>>
>>> Mladen: It is going to be in house, not on cloud. and 1.5Tb LUNs are
>>> common (with SSDs?)? How is the IO in those installations? I understand
>>> that SSDs outperform HDDs, but I am wondering, just wondering, that by
>>> providing 2 or 4 luns we are losing the advantage that SSDs give us?
>>>
>>>
>>> On Thu, Oct 19, 2017 at 1:39 AM, Jonathan Lewis <
>>> jonathan_at_jlcomp.demon.co.uk> wrote:
>>>
>>>>
>>>> At some layer between Oracle and the silicon the various software
>>>> components will have some queues. If there is a layer at which you have a
>>>> single queue to each LUN you will have an I/O bottleneck when you've got
>>>> lots of Oracle processes trying to read from just 2 (or 4) LUNs.
>>>>
>>>> I'm not an expert with stuff that far away from the Oracle software but
>>>> I would be a little surprised if you got bad performance because you were
>>>> configured as 40 LUNs, while I have seen bad performance from a system
>>>> where the solid state SAN had been configured as just 2 LUNs (one for data,
>>>> one for redo).
>>>>
>>>>
>>>> Regards
>>>> Jonathan Lewis
>>>>
>>>> ________________________________________
>>>> From: oracle-l-bounce_at_freelists.org <oracle-l-bounce_at_freelists.org> on
>>>> behalf of Ram Raman <veeeraman_at_gmail.com>
>>>> Sent: 19 October 2017 06:49:23
>>>> To: ORACLE-L
>>>> Subject: SSDs and LUNs
>>>>
>>>> We are moving one of the systems to vm. The consultants who have been
>>>> hired to do the implementation are recommending that we create just 2 or 4
>>>> 'LUNS' for data diskgroup for the db that is 3Tb in size which exhibits
>>>> hybrid IO. They are promising it is best rather than having 30 or 40 LUNs
>>>> since the new disks will all be SSDs.They are claiming that it will perform
>>>> better than having 40 'LUNs'. I still have the 'old way of thinking' when
>>>> it comes to IO. Can someone confirm one way or other, or point to any
>>>> paper. thanks.
>>>>
>>>> Ram.
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>
>>
>> --
>> Niall Litchfield
>> Oracle DBA
>> http://www.orawin.info
>>
>
>
>
> --
>
>
>

-- 
Niall Litchfield
Oracle DBA
http://www.orawin.info

--
http://www.freelists.org/webpage/oracle-l