Re: Slower performance after enabling async io on Oracle Linux
From: Matthias Hoys <matthias.hoys_at_gmail.com>
Date: Tue, 15 May 2012 07:07:47 -0700 (PDT)
Message-ID: <29307150.2391.1337090867550.JavaMail.geo-discussion-forums_at_vbvq7>
On Tuesday, May 15, 2012 3:16:46 PM UTC+2, Matthias Hoys wrote:
> On Tuesday, May 15, 2012 2:52:33 AM UTC+2, Mladen Gogala wrote:
> > Matthias, I the problem may as well lie on the network side. Check netstat
> > -s for timeouts, retransmits or packets being dropped. Depending on the
> > hardware, you may want to use jumbo packets, increase rmem and wmem
> > parameters. Below are two articles about tuning the network:
> >
> > http://fasterdata.es.net/host-tuning/linux/
> > http://www.ibm.com/developerworks/linux/library/l-adfly/index.html
> >
> > Problem with Linux is that it is not well instrumented, there is no wait
> > event interface so you cannot use the same technique as with the
> > database. You can use systemtap to explore the performance of the network
> > layer and figure out what is taking so long.
> > Also, hopefully your iSCSI traffic doesn't have to go through any
> > firewall? Packet inspection can take time, just as with the customs
> > office.
> >
> > So, if you want a recipe, measure the duration of the system calls by
> > using strace and systemtap. Check netstat -s, make sure that your network
> > is optimally configured.
> >
>
> Using netstat -s, I think I found something:
>
> Tcp:
> 1168305 active connections openings
> 264105 passive connection openings
> 910210 failed connection attempts
>
> I'm no network specialist, but it seems like there are a huge amount of "failed connection attempts"?? Or is this normal for iSCSI, I never worked with iSCSI storage before...
>
> Btw: we don't have a local disk, everything is on VMWare and the disks are on the iSCSI box.
>
> Thanks for the help,
> Matthias Hoys
Date: Tue, 15 May 2012 07:07:47 -0700 (PDT)
Message-ID: <29307150.2391.1337090867550.JavaMail.geo-discussion-forums_at_vbvq7>
On Tuesday, May 15, 2012 3:16:46 PM UTC+2, Matthias Hoys wrote:
> On Tuesday, May 15, 2012 2:52:33 AM UTC+2, Mladen Gogala wrote:
> > Matthias, I the problem may as well lie on the network side. Check netstat
> > -s for timeouts, retransmits or packets being dropped. Depending on the
> > hardware, you may want to use jumbo packets, increase rmem and wmem
> > parameters. Below are two articles about tuning the network:
> >
> > http://fasterdata.es.net/host-tuning/linux/
> > http://www.ibm.com/developerworks/linux/library/l-adfly/index.html
> >
> > Problem with Linux is that it is not well instrumented, there is no wait
> > event interface so you cannot use the same technique as with the
> > database. You can use systemtap to explore the performance of the network
> > layer and figure out what is taking so long.
> > Also, hopefully your iSCSI traffic doesn't have to go through any
> > firewall? Packet inspection can take time, just as with the customs
> > office.
> >
> > So, if you want a recipe, measure the duration of the system calls by
> > using strace and systemtap. Check netstat -s, make sure that your network
> > is optimally configured.
> >
>
> Using netstat -s, I think I found something:
>
> Tcp:
> 1168305 active connections openings
> 264105 passive connection openings
> 910210 failed connection attempts
>
> I'm no network specialist, but it seems like there are a huge amount of "failed connection attempts"?? Or is this normal for iSCSI, I never worked with iSCSI storage before...
>
> Btw: we don't have a local disk, everything is on VMWare and the disks are on the iSCSI box.
>
> Thanks for the help,
> Matthias Hoys
Update: the Oracle Listener is causing all the "failed connection attempts". When I stop it, no more failed connection attempts. I'm now further debugging this...
Matthias Received on Tue May 15 2012 - 09:07:47 CDT