Tru64 - Linux transfers

From: Herring Dave - dherri <>
Date: Tue, 24 Mar 2009 10:47:53 -0500
Message-ID: <>


I have a need to transfer a large amount of data between a Tru64 server (let's call it NODE_TRU) and a Linux server (NODE_LIN) and am finding a huge difference in pulling in one direction vs. the other.

Before messing around with any Oracle Net settings, I thought I'd first validate transfer rates node-to-node, using sftp. NODE_TRU is running Tru64 5.1 and is a 64bit GS1280 4 x 16, using SAN attached storage. Concurrent transfers were tested using 500 MB files on separate filesystems, each created using "dd" off /dev/zero. NODE_LIN is running RHEL 4.1 and is a 64bit 3850 quadcore 2 x 32, using SAN attached storage (EMC raw with ASM). The connection between the 2 is a Gb line. Here are the results of my testing so far, using 1, 2, 3, and 4 concurrent transfers:

					Concurrent Transfer Results (MB/s)					
sftp direction	      	1	2	3	4		
NODE_TRU Get from NODE_LIN	 4.33	 8.66	12.30	15.04		
NODE_TRU Put to NODE_LIN	 6.33	12.20	16.67	18.87	 	 37.02% Avg % faster than NODE_TRU "Get" from NODE_LIN
NODE_LIN Get from NODE_TRU	14.70	27.41	35.71	38.52		200.60% Avg % faster than NODE_TRU "Get" from NODE_LIN
NODE_LIN Put to NODE_TRU	 4.10	 8.13	12.10	16.39		-27.29% Avg % slower than NODE_TRU "Put" to NODE_LIN

On NODE_TRU we increased tcp_sendspace and tcp_recvspace from 61,440 to 1,048,576. On NODE_LIN we increased net.ipv4.tcp_rmem from 4096 87380 174760 to 4096 87380 16777216, net.ipv4.tcp_wmem from 4096 16384 131072 to 4096 65536 16777216, and both net.core.rmem_max and net.core.wmem_max from 262,144 to 16,777,216. The above tests were done after these parameter changes and unfortunately I only tested NODE_TRU Put to NODE_LIN before these changes (no real differences found in this transfer with the changes in place).

Are those the appropriate kernel changes to basically open the flood gates between the 2 servers? This is all in preparation for a huge transfer from NODE_TRU to NODE_LIN, so any increase in transfer rate will help greatly.

Thanks in advance for any helpful insight!

David C. Herring  | DBA, Acxiom Automotive

630-944-4762 office | 630-430-5988 cell | 630-944-4989 fax 1501 Opus Pl | Downers Grove, IL, 60515 | U.S.A. |

The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged.

If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.

If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system.

Thank You.

Received on Tue Mar 24 2009 - 10:47:53 CDT

Original text of this message