Moved Apache web server to vmware from ganetti ... network waits abound (Some strace info)

From: Chris Taylor <christopherdtaylor1994_at_gmail.com>
Date: Wed, 29 Aug 2018 14:39:03 -0500
Message-ID: <CAP79kiQF02yx9updZYY1uTvjTOwN+z7Te1bad=A1NkcvVoiDTQ_at_mail.gmail.com>

We recently moved our Apache web servers to vmware from ganetti and basically cloned everything.

However, we're getting weird waits from the Apache server. Here's what an strace looks like FROM the database server:

The read(14 is the network connection to the Apache server. The Event in the database is "SQL*Net MORE Data From Client"

A big, big object is built on the apache side and passed over the network to the database. This object is broken up into chunks.

When on Ganetti we didn't see these network waits... each chunk would come across pretty fast. But now, there's a significant wait for each chunk.

I'm looking to see if anyone had "tune" Apache/VMWare configurations talking across SQLNet to Oracle and if so, what did you tweak? We've changed the network card from E1000 to VMXnet (something, something) and its bridged, not NAT. I think that improved things somewhat but still lots waits between each READ from CLIENT connection.

So each read(14 is reading the open network connection to the Apache server. Used lsof to find what "14" was here. The web/app server has built a very large object that we store in a LOB The object is base64 encoded before it is passed to Oracle

120.304233 read(14, "stripped", 553) = 553

120.319649 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 215967},

ru_stime={0, 94985}, ru_maxrss=27116, ru_ixrss=0, ru_idrss=0, ru_isrss=0,
ru_minflt=9792, ru_majflt=0, ru_nswap=0, ru_inblock=896, ru_oublock=0,
ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=285, ru_nivcsw=17}) = 0
     0.000619 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 215967},
ru_stime={0, 94985}, ru_maxrss=27116, ru_ixrss=0, ru_idrss=0, ru_isrss=0,

ru_minflt=9792, ru_majflt=0, ru_nswap=0, ru_inblock=896, ru_oublock=0, ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=287, ru_nivcsw=17}) = 0

0.000038 read(14, "stripped", 8015) = 2920 120.319281 read(14, "stripped", 5095) = 2920 120.324230 read(14, "stripped", 2175) = 2175

120.316067 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 215967},

ru_stime={0, 95985}, ru_maxrss=27116, ru_ixrss=0, ru_idrss=0, ru_isrss=0,
ru_minflt=9792, ru_majflt=0, ru_nswap=0, ru_inblock=896, ru_oublock=0,
ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=312, ru_nivcsw=17}) = 0

0.000274 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 215967},

ru_stime={0, 95985}, ru_maxrss=27116, ru_ixrss=0, ru_idrss=0, ru_isrss=0,
ru_minflt=9792, ru_majflt=0, ru_nswap=0, ru_inblock=896, ru_oublock=0,
ru_msgsnd=0, ru_msgrcv=0, ru_nsignals=0, ru_nvcsw=314, ru_nivcsw=17}) = 0

--
http://www.freelists.org/webpage/oracle-l

Received on Wed Aug 29 2018 - 21:39:03 CEST