Re: ISCSI EMC OEL performance issue

From: Gaja Krishna Vaidyanatha <>
Date: Wed, 7 Mar 2012 10:17:05 -0800 (PST)
Message-ID: <>

Hi Vamshi,
Hope you are doing well. Storage Performance Troubleshooting in a SAN is like unraveling a 7-layer burrito....:) There are many layers of abstraction in a SAN and the concept of "throughput measurement" (in your case data transfer rates) has to be done keeping the "lowest common denominator". What I mean here is this - Even if your storage volume can push 1024MB/sec in data transfers, your net throughput can be limited by the other parts in your SAN (i.e. switches/HBAs/bus etc). If your switch/HBA supports 6Gb/sec, the maximum theoretical throughput you can achieve is 768 MB/sec. This is because the "b" in "6Gb" is a small b, standing for Gigabit, so one has to do the relevant arithmetic (divide by 8) for this. I pulled up the specs for your storage array -, but it does not quite tell me what this beast can actually achieve from a data transfer standpoint. May be I have  missed something in the spec. It may not be a bad idea to sit down with your EMC storage folks and the internal storage team, layout the physical architecture and tag some numbers on the data transfer rates of each component. This will then give you the true theoretical limit for data transfer. And that could be a better starting point to set expectations on what your SAN can deliver. Let me know if you have any further questions.  


Gaja Krishna Vaidyanatha,
CEO & Founder, DBPerfMan LLC

Phone - +1-650-743-6060 Insights:Tales of the Oak Table - Co-author:Oracle Performance Tuning 101 - Enabling Cloud Deployment & Management for Oracle Databases

 From: Vamshi Damidi <> To:
Sent: Tuesday, March 6, 2012 8:19 AM
Subject: ISCSI EMC OEL performance issue  

Hi All,
Hope every one is doing great, busy with research. Any suggestions to this post is appreciated.

DESC: We have bought new SAN EMC VNX 5300 / OEL / Broadcom CNA 10G / ISCI Lun Size 5**G
Disk : 15k RPM * 13 disks

Current IO calibaration  results :


------------------------------ ------------------------------ ----------
---------- ---------- ---------- ------------------
28-FEB-12 PM  28-FEB-12 PM 4148        433        295        11                24

i observe that actual is not match or near to calculated. Can you please suggest some thoughts for imporvment.

[root_at_srvpdb01 ~]# cat /etc/iscsi/iscsid.conf

node.startup = automatic
node.session.timeo.replacement_timeout = 120
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.noop_out_interval = 10
node.conn[0].timeo.noop_out_timeout = 15
node.session.initial_login_retry_max = 4
node.session.cmds_max = 128
node.session.queue_depth = 128
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 16776192
node.session.iscsi.MaxBurstLength = 16776192

# the default is 131072

node.conn[0].iscsi.MaxRecvDataSegmentLength = 16776192
# the default is 32768

discovery.sendtargets.iscsi.MaxRecvDataSegmentLength =  16776192 node.conn[0].iscsi.HeaderDigest = None
node.session.iscsi.FastAbort = No
net.core.rmem_max = 1073741824
net.core.wmem_max = 1073741824
net.ipv4.tcp_rmem = 1048576 16777216 1073741824
net.ipv4.tcp_wmem = 1048576 16770216 1073741824
net.ipv4.tcp_mem = 1048576 16770216 1073741824

Above are my current operational parameters.

Vamshi .D


Received on Wed Mar 07 2012 - 12:17:05 CST

Original text of this message