Return-Path: X-Original-To: oracle-l@orafaq.com Delivered-To: oracle-l@orafaq.com Received: from puck1183.startdedicated.com (localhost [127.0.0.1]) by puck1183.startdedicated.com (Postfix) with ESMTP id AE9E31960732 for ; Fri, 11 Nov 2016 21:52:09 +0100 (CET) Received: from turing.freelists.org (turing.freelists.org [206.53.239.180]) by puck1183.startdedicated.com (Postfix) with ESMTPS for ; Fri, 11 Nov 2016 21:52:09 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 5085C620AB; Fri, 11 Nov 2016 15:52:08 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at turing.freelists.org Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F26vEfmpoMmR; Fri, 11 Nov 2016 15:52:08 -0500 (EST) Received: from turing.freelists.org (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 93ED1620A6; Fri, 11 Nov 2016 15:51:55 -0500 (EST) Received: with ECARTIS (v1.0.0; list oracle-l); Fri, 11 Nov 2016 15:50:33 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id BF05962084 for ; Fri, 11 Nov 2016 15:50:33 -0500 (EST) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DxE-VChTIa_y for ; Fri, 11 Nov 2016 15:50:33 -0500 (EST) Received: from mail-it0-f51.google.com (mail-it0-f51.google.com [209.85.214.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 89EFD62080 for ; Fri, 11 Nov 2016 15:50:33 -0500 (EST) Received: by mail-it0-f51.google.com with SMTP id e187so347828130itc.0 for ; Fri, 11 Nov 2016 12:50:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=LJdwr7bUBzdmzhbXZGHOkOBfqCd83sOi+JFbQMX0Tvw=; b=f7N0ImDH4vE4Xum3S3buWMqfgu2wJd5gimCoMAQrsVzuarmeiC4S4myhTz2EKvZ7bY DEhSfkEaAbF520GoddroBFdEY4Buf6OB1li0P6iF2M8/V0X382UR0Ux7kedxDCL8J7d2 IhWEFyu4qkNlrhl2to4GbzqjOmqq7zRgCzYIatoEDpH7qO4weF4BgzkVUhgvAx6aIHDv pdYdhIqUmzGHjSR7++jsnf7/4MJmCr4eDf2UqJHuhg6neT+AoHXWn5nlvj1G67hig4nw PB4lZ3asFiOd4BRTtx1oZsOlq0FvYQh+AcQs8JGlVIDjL1CRTFQzW+4UnKy/0b8WJer+ Wy0Q== X-Gm-Message-State: ABUngveos5OuZeolZgXJ/JT6KDAFjg4Xfb40+InHIXKiKbl7AWbWUHocxm6qSRU6VUe3Fc+Mr5rkBqRRX4NaHQ== X-Received: by 10.157.61.98 with SMTP id a89mr1670635otc.86.1478897430573; Fri, 11 Nov 2016 12:50:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.63.100 with HTTP; Fri, 11 Nov 2016 12:50:30 -0800 (PST) In-Reply-To: References: <441952295.2260945.1478884537701@mail.yahoo.com> <012601d23c57$9cd51fc0$d67f5f40$@comcast.net> From: Jeff Chirco Date: Fri, 11 Nov 2016 12:50:30 -0800 Message-ID: Subject: Re: jumbo frames with non rac To: tim.evdbt@gmail.com Cc: Dimensional DBA , alfredo.abate@gmail.com, kathy duret , "oracle-l@freelists.org" Content-Type: multipart/alternative; boundary=001a11c01db002690605410ca45e X-archive-position: 66825 X-ecartis-version: Ecartis v1.0.0 Sender: oracle-l-bounce@freelists.org Errors-to: oracle-l-bounce@freelists.org X-original-sender: backseatdba@gmail.com Precedence: normal Reply-To: backseatdba@gmail.com List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: oracle-l X-List-ID: oracle-l List-subscribe: List-owner: List-post: List-archive: X-list: oracle-l --001a11c01db002690605410ca45e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Tim well point. I think at this point for us it is more of a proactive preventive measure. We are in the process of moving our database servers from Windows to Linux so I say why not just go with jumbo frames now if it is not too much cost and trouble especially with all the testing we will be doing for the OS conversion. Better than trying to do it later. Jeff On Fri, Nov 11, 2016 at 12:44 PM, Tim Gorman wrote: > To augment what both Matthew and Alfredo have said... > > TCP "jumbo frames" are used mainly with network-attached storage (i.e. > NFS, iSCSI, etc) and for special use-cases like the Oracle RAC > interconnect. The standard 1500-byte packets are sized to optimally > accommodate network traffic ranging from acknowledgement messages of a fe= w > bytes to database storage traffic of 8192 bytes and larger. > > Larger packet sizes doesn't make the packets move any faster over the wir= e > or through switches and routers, but simply results in fewer packets > because an 8192-byte database block fits within one 9000-byte packet, > instead of requiring six 1500-byte packets. This results in fewer > un-marshalling operations on the source server and fewer marshalling > operations on the destination server, so the ultimate increase in network > throughput actually comes from less CPU consumption by the servers on > either end of the network connection. The CPU savings on each server, as > well as the network throughput, can be significant, but when there is a > consistently high volume of such large-packet traffic. In low-volume > situations, it is difficult to measure any benefit. > > CPU-saturated servers using network-attached storage without jumbo frames > can result in poor I/O performance without network saturation being > detectable. DBAs will report the poor I/O latency and throughput over NF= S > storage, but the network administrator will correctly respond that there = is > no network throughput or latency issues. Meanwhile, server administrator= s > will surely detect the CPU saturation, but not associate it with the poor > I/O performance over NAS, likely because they won't be involved in the I/= O > performance discussion, and neither the DBA nor the network admin will ma= ke > the connection. > > So, deploying jumbo frames can be a reactive measure when CPU saturation > is detected and associated to NAS I/O issues, or it can be a proactive > measure to prevent that situation. > > > > > > On 11/11/16 13:10, Dimensional DBA wrote: > > As Alfredo mentions you normally don=E2=80=99t use jumbo frames on the pu= blic > facing networks as none of the clients normally have jumbo frames setup. > This could be different if you were using say citrix desktops and ewveryo= ne > used those to access their applications then what you want may work. > > I have also used jumbo frames on storage networks to netapps or other > network storage. > > > > *Matthew Parker* > > *Chief Technologist* > > *Dimensional DBA* > > *425-891-7934 <425-891-7934> (cell)* > > *D&B *047931344 > > *CAGE *7J5S7 > > *Dimensional.dba@comcast.net * > > *View Matthew Parker's profile on LinkedIn* > > > www.dimensionaldba.com > > > > *From:* oracle-l-bounce@freelists.org [mailto:oracle-l-bounce@ > freelists.org ] *On Behalf Of *Alfredo > Abate > *Sent:* Friday, November 11, 2016 11:55 AM > *To:* Jeff Chirco > *Cc:* kathy duret; oracle-l@freelists.org > *Subject:* Re: jumbo frames with non rac > > > > Jeff, > > > > My understanding of using jumbo frames with Oracle RAC is to be able to > increase the packet size of the *private interconnect* (heartbeat) of the > cluster for better performance. This is normally done on segregated > switches (or switch ports), NICs, etc so that they are all configured for > the larger MTU size (9000). > > > > If you are considering enabling jumbo frames on the *public network* of a > single instance (or even RAC) database server that your application serve= rs > or end users will directly be connecting to it can cause issues since mos= t > networks are configured for the normal MTU size (1500). This mismatch wi= ll > potentially cause network issues such as packet drops when the two device= s > are communicating with each other. > > > > Alfredo > > > > On Fri, Nov 11, 2016 at 12:38 PM, Jeff Chirco > wrote: > > Sorry yes Oracle Linux 7 here and with hugepages. 11g database currently > but plans to 12c. > > > > On Fri, Nov 11, 2016 at 9:15 AM, kathy duret > wrote: > > I am assuming Linux here ... > > > > You will also need to set up hugepages and there are some other settings > to consider like semaphores. > > > > There are many papers on how to do set this up. I would look on MOS firs= t > and then go fro there. > > > > *Kathy Duret* > > > > > ------------------------------ > > *From:* Jeff Chirco > *To:* "oracle-l@freelists.org" > *Sent:* Friday, November 11, 2016 10:30 AM > *Subject:* jumbo frames with non rac > > > > We are in the middle of setting up replacing new database servers and bee= n > wondering about jumbo frames. We have 10gb network and we don't run RAC > and not sure if we ever will but wondering if there still is a big benefi= t > for jumbo frames? We also are running a new NetApp all flash storage. > > From what I have read so far is yes there is a benefit but just wondering > if anyone has any opinions? I am not very knowledgeable on the network si= de > of things. > > Thanks, > > Jeff > > > > > > > > > --001a11c01db002690605410ca45e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Tim well point.=C2=A0 I think at this point fo= r us it is more of a proactive preventive measure.=C2=A0 We are in the proc= ess of moving our database servers from Windows to Linux so I say why not j= ust go with jumbo frames now if it is not too much cost and trouble especia= lly with all the testing we will be doing for the OS conversion.=C2=A0 Bett= er than trying to do it later.

Jeff

On Fri, Nov 11, 2016 at 12:44 PM, = Tim Gorman <tim.evdbt@gmail.com> wrote:
=20 =20 =20
To augment what both Matthew and Alfredo have said...

TCP "jumbo frames" are used mainly with network-attached stor= age (i.e. NFS, iSCSI, etc) and for special use-cases like the Oracle RAC interconnect.=C2=A0 The standard 1500-byte packets are sized to optimal= ly accommodate network traffic ranging from acknowledgement messages of a few bytes to database storage traffic of 8192 bytes and larger.

Larger packet sizes doesn't make the packets move any faster over the wire or through switches and routers, but simply results in fewer packets because an 8192-byte database block fits within one 9000-byte packet, instead of requiring six 1500-byte packets.=C2=A0 Thi= s results in fewer un-marshalling operations on the source server and fewer marshalling operations on the destination server, so the ultimate increase in network throughput actually comes from less CPU consumption by the servers on either end of the network connection.=C2= =A0 The CPU savings on each server, as well as the network throughput, can be significant, but when there is a consistently high volume of such large-packet traffic.=C2=A0 In low-volume situations, it is difficult to measure any benefit.

CPU-saturated servers using network-attached storage without jumbo frames can result in poor I/O performance without network saturation being detectable.=C2=A0 DBAs will report the poor I/O latency and throughput over NFS storage, but the network administrator will correctly respond that there is no network throughput or latency issues.=C2=A0 Meanwhile, server administrators will surely detect the C= PU saturation, but not associate it with the poor I/O performance over NAS, likely because they won't be involved in the I/O performance discussion, and neither the DBA nor the network admin will make the connection.

So, deploying jumbo frames can be a reactive measure when CPU saturation is detected and associated to NAS I/O issues, or it can be a proactive measure to prevent that situation.





On 11/11/16 13:10,= Dimensional DBA wrote:
=20 =20 =20 =20

As Alfredo mentions you normally don=E2=80=99t use jumbo frames on= the public facing networks as none of the clients normally have jumbo frames setup. This could be different if you were using say citrix desktops and ewveryone used those to access their applications then what you want may work.

I have also used jumbo frames on storage networks to netapps or other network storage.

=C2=A0<= /u>

Matthew Parker

Chief Technologist

Dimensional DBA

425-891-7934 (cell)

D&B 047931344

CAGE 7J5S7

Dimensional.dba@comcast.ne= t

View Matthew Parker's profile on LinkedIn

www.dimensionaldba.com

=C2=A0<= /u>

From: oracle-l-bounc= e@freelists.org [mailto:oracle-l= -bounce@freelists.org] On Behalf Of Alfredo Abate
Sent: Friday, November 11, 2016 11:55 AM
To: Jeff Chirco
Cc: kathy duret; oracle-l@freelists.org
Subject: Re: jumbo frames with non rac

=C2=A0

Jeff,

=C2=A0

My understanding of using jumbo frames with Oracle RAC is to be able to increase the packet size of the private interconnect (heartbeat) of the cluster for better performance.=C2=A0 This is normally do= ne on segregated switches (or switch ports), NICs, etc so that they are all configured for the larger MTU size (9000).

=C2=A0

If you are considering enabling jumbo frames on the public network of a single instance (or even RAC) database server that your application servers or end users will directly be connecting to it can cause issues since most networks are configured for the normal MTU size (1500).=C2=A0 This mismatch will potentially cause network issues such as packet drops when the two devices are communicating with each other.

=C2=A0

Alfredo

=C2=A0

On Fri, Nov 11, 2016 at 12:38 PM, Jeff Chirco <backseatdba@gmail.com> wrote:

Sorry yes Oracle Linux 7 here and with hugepages.=C2=A0 11g database currently but plans to 12c.

=C2=A0

On Fri, Nov 11, 2016 at 9:15 AM, kathy duret <katpopins21@yahoo.com> wrote:

I am assuming Linux here ...

=C2=A0

You will also need to set up hugepages and there are some other settings to consider like semaphores.=C2=A0

=C2=A0

There are many papers on how to do set this up.=C2=A0 I would look on MOS first and then go fro there.=

=C2=A0

<= b>Kathy Duret

<= b>=C2=A0

=C2=A0

From: Jeff Chirco <backseatdba@gmail.com>
To: "oracle-l@freelists.org" <oracle-l@freelists.org>
Sent: Friday, November 11, 2016 10:30 AM
Subject: jumbo frames with non rac

=C2=A0

We are in the middle of setting up replacing new database servers and been wondering about jumbo frames.=C2=A0 We have 10gb network and we don't run RAC and not sure if we ever will but wondering if there still is a big benefit for jumbo frames? We also are running a new NetApp all flash storage.

From what I have read so far is yes there is a benefit but just wondering if anyone has any opinions? I am not very knowledgeable on the network side of things.

Thanks,

Jeff

=C2=A0

=C2=A0

=C2=A0



--001a11c01db002690605410ca45e-- -- http://www.freelists.org/webpage/oracle-l