Return-Path: <oracle-l-bounce@freelists.org>
X-Original-To: oracle-l@orafaq.com
Delivered-To: oracle-l@orafaq.com
Received: from smtp-aa.freelists.org (smtp-aa.freelists.org [23.23.80.81])
 by malta2546.startdedicated.com (Postfix) with ESMTPS id 39AAE10036E784
 for <oracle-l@orafaq.com>; Tue,  2 Feb 2021 11:24:00 +0100 (CET)
Received: from turing.freelists.org (ip-10-0-0-164.ec2.internal [10.0.0.164])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits))
 (No client certificate requested)
 by smtp-aa.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 3EAD844AA1;
 Tue,  2 Feb 2021 10:23:59 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by turing.freelists.org (Postfix) with ESMTP id 29E613F85E;
 Tue,  2 Feb 2021 10:23:59 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=freelists.org;
 s=turing; t=1612261439;
 bh=QhOppd0gXZK6zXPCEksR181VMdrzeEsWRLvw7GoKbxI=;
 h=From:Sender:Sender:From;
 b=ctEPSxAsixvp54YYbQ/WUWjQg5QUoCIQje+H1DStl1TE5kQqLCI44n7BFvivauk7f
	 AG/4JSemI77IemapoE3Q3owPJ730Vfb1uM+EmIW2MRJpfV4F/+FR9yW5cFAhRw/rjo
	 EYul4kgPoB+1e7IxQmFxixIjFRi/gmKH3BZyvdjo=
X-Virus-Scanned: by FreeLists at turing2.freelists.org
Received: from turing.freelists.org ([127.0.0.1])
 by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 8e19HQjfmq3p; Tue,  2 Feb 2021 10:23:59 +0000 (UTC)
Received: from turing.freelists.org (localhost [127.0.0.1])
 by turing.freelists.org (Postfix) with ESMTP id 57EFD3F970;
 Tue,  2 Feb 2021 10:23:56 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=freelists.org;
 s=turing; t=1612261437;
 bh=QhOppd0gXZK6zXPCEksR181VMdrzeEsWRLvw7GoKbxI=;
 h=From:Sender:Sender:From;
 b=RofAy6JLxBiaBk85uX8vSjkzdTxVpHC0CPAT1g0C2dwUABB3hmcxEJkMq9lSQEY7j
	 DbNXLR3I7N4I95QF95pgE/HC9BgTa4y5XLkGwCbpDjV5wvS2a2/ZtYkB61LcUPdMam
	 yYGAyAHWsvY8fWCJ3Aq3eYAEXoiArpB5YUHWNVlA=
Received: with ECARTIS (v1.0.0; list oracle-l); Tue, 02 Feb 2021 10:23:55 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by turing.freelists.org (Postfix) with ESMTP id 1C0043F85E
 for <oracle-l@freelists.org>; Tue,  2 Feb 2021 10:23:55 +0000 (UTC)
Authentication-Results: turing.freelists.org;
 dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=B5EJUOj9;
 dkim-atps=neutral
Received: from turing.freelists.org ([127.0.0.1])
 by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 4_3EPkjGh6CN for <oracle-l@freelists.org>;
 Tue,  2 Feb 2021 10:23:55 +0000 (UTC)
Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by turing.freelists.org (Postfix) with ESMTPS id 029F13F844
 for <oracle-l@freelists.org>; Tue,  2 Feb 2021 10:23:54 +0000 (UTC)
Received: by mail-io1-f44.google.com with SMTP id u8so15587647ior.13
        for <oracle-l@freelists.org>; Tue, 02 Feb 2021 02:23:54 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to:cc;
        bh=9xGOhiPdaHyplrdXm9weyxD1mclxI0UxLObaZGkhu5g=;
        b=WZcMOBOhE+GyZy/b05QgeEztV1uY/VV9i1g2vYBxXHWWZV8MAu1LR8MiA1QzZZSegY
         qLi6iZsYv2DqmF9l6mqnMCkz1OJVffWhJaJnhfZd0LGVRktYrn7GneBpVqtTDhahSgum
         3RcWNkFxOHo3bh1ohE7QYUl+YjX7ByWRh0TPVSG1xlqQprdUGFsjmWW1YiLa0ctRzhFM
         nn/tMO31MjcnmECG8eAvU+5Oz7OhJAeXOdeMgzfbVV57GDdiqIs7yqOnHBl/JCH6D2W8
         N4xfc8eEeBa0YFIMRwCmMdjrFJOqZpVwYnP8T9Sw8Qk5C7YeXgS/2+/bn4ts30tAb7A6
         fRZA==
X-Gm-Message-State: AOAM533vmUnfBCJEQwziI819+I2J97Q5t1/9oWOnAUyGDJrW6zQ2KFmE
 VjUto45fVNbx+XZC57O2CPU71+XDdn4cLcQoWT4=
X-Google-Smtp-Source: ABdhPJyKsRyZuuyEP6zHIIokXiHaZK20l2DS+lawQ17sJHe5V9vAJ6DzHRvehY0VpYYubtwzYbdsbdTDW2+jY4PHSMA=
X-Received: by 2002:a05:6638:40b:: with SMTP id q11mr1923476jap.133.1612261434203;
 Tue, 02 Feb 2021 02:23:54 -0800 (PST)
MIME-Version: 1.0
References: <CAEjw_fg5yfOWT2KOkYRn+pqkMhgRJC+cQnuSPO+Uz5s2m9DnBA@mail.gmail.com>
 <CAOVevU5ZH2_znOXdVLid445nC-c5vZRddoDSZ+tDMUPM1RKz7Q@mail.gmail.com>
 <CA+riqSXW1K7X+B2wF-=tT0mxPncTRrtKt28KqBJrstnypb6TyQ@mail.gmail.com>
 <CAEjw_fhO-VurdrgCLxse=s4r-CxmRtVGpBVvD+nCNmUrT3inMg@mail.gmail.com>
 <CAEfe=X_DTUd6bbY8VuQ4ZUrR89oHQm0e_H0w=vzrP7YwXkRjQQ@mail.gmail.com>
 <CAJu8R6iDhUQv=uPN_UsgLd_opCU+Q4+T-jUzd_qd-=xp6f-JzA@mail.gmail.com>
 <CAEjw_fhwEGLh8tFzfs5hfODx61u+yk7JAb6XAxB0k-KCcO5V3A@mail.gmail.com>
 <CA+riqSXZ8pYhmM4c7smzwK4H=pJaRaOmBAVLOJ4nfqmY9g-q1A@mail.gmail.com> <CAJu8R6jXi-QodM-BzKhDOSw0Ji6XWNuSp0BpVtivGizSY8Ng3w@mail.gmail.com>
In-Reply-To: <CAJu8R6jXi-QodM-BzKhDOSw0Ji6XWNuSp0BpVtivGizSY8Ng3w@mail.gmail.com>
From: Laurentiu Oprea <laurentiu.oprea06@gmail.com>
Date: Tue, 2 Feb 2021 12:23:10 +0200
Message-ID: <CA+riqSUVKX7x5P_ENCGzJ51WEb07=qiyQeS_LZ0CZVXJCCBQKQ@mail.gmail.com>
Subject: Re: Library lock issue
To: Mohamed Houri <mohamed.houri@gmail.com>
Cc: Pap <oracle.developer35@gmail.com>, Chinar Aliyev <chinaraliyev@gmail.com>,
 Sayan Malakshinov <xt.and.r@gmail.com>, Oracle L <oracle-l@freelists.org>
Content-Type: multipart/alternative; boundary="00000000000012223105ba57de0b"
X-archive-position: 78822
X-ecartis-version: Ecartis v1.0.0
Sender: oracle-l-bounce@freelists.org
Errors-to: oracle-l-bounce@freelists.org
X-original-sender: laurentiu.oprea06@gmail.com
Precedence: normal
Reply-To: laurentiu.oprea06@gmail.com
List-Help: <mailto:ecartis@freelists.org?Subject=help>
List-Unsubscribe: <mailto:oracle-l-request@freelists.org?Subject=unsubscribe>
List-software: Ecartis version 1.0.0
List-Id: oracle-l <oracle-l.freelists.org>
X-List-ID: oracle-l <oracle-l.freelists.org>
List-Subscribe: <mailto:oracle-l-request@freelists.org?Subject=subscribe>
List-Owner: <mailto:>
List-post: <mailto:oracle-l@freelists.org>
List-Archive: <http://www.freelists.org/archives/oracle-l>
X-list: oracle-l
--00000000000012223105ba57de0b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Nice one, thanks Mohamed. I`ll update on this thread once I reach the
bottom of my issues as well.

=C3=8En mar., 2 feb. 2021 la 12:19, Mohamed Houri <mohamed.houri@gmail.com>=
 a
scris:

> @Laurentiu
>
> As I mentioned above it was a kind of bug or kind of =E2=80=9Cwe don=E2=
=80=99t know why
> when using bind variable PX slaves were refusing to share the execution
> plan of their QC=E2=80=9D. I have had an excellent private discussion wit=
h Oracle
> support engineer about this issue and he clearly explained that this was
> reported to the CBO team.
>
> So, I am sorry I haven=E2=80=99t any bug to share with you for this issue=
.
> Moreover, it happened in 12cR1 and, as I said above, I still have not
> encountered a similar issue in 12cR2, 18c, and 19c where I am using
> parallelism extensively
>
> @Pap
>
> My answer was that, *probably*, your parallel parsing issue is due to the
> usage of GTT private statistics. So if your planned fix tentative will
> reveal to be useless, I would then suggest checking if using GTT shared
> statistics will get rid of the library cache lock and cursor pin S wait o=
n
> x wait events. If so then you have to balance between sharing inadequate
> GTT statistics and parsing effects.
>
>
>
> *=E2=80=9CAll things being equal you must balance between the performance
> improvement brought by this new GTT feature and the parsing side effect i=
t
> introduces because of the underlying cursor invalidation. In my client
> case,** the Library cache and Cursor Pin S wait on X wait events
> introduced by the SESSION PRIVATE statistics largely outweigh the
> performance penalty that comes when the 49 streams share the same GTT
> statistics.=E2=80=9D*
>
>
>
> Best regards
>
> Mohamed Houri
>
> Le mar. 2 f=C3=A9vr. 2021 =C3=A0 11:10, Laurentiu Oprea <laurentiu.oprea0=
6@gmail.com>
> a =C3=A9crit :
>
>> Hello Pap,
>>
>> I was mentioning DS in the context of possible bug as well. If L2 is use=
d
>> most probably you don't have an issue around this and your issue might b=
e
>> around GTTs stats.
>>
>> =C3=8En mar., 2 feb. 2021 la 11:57, Pap <oracle.developer35@gmail.com> a
>> scris:
>>
>>> Thanks much.
>>>
>>> For that query , I see the below note section. And as i mentioned are
>>> using private session level stats collected for two of the global tempo=
rary
>>> tables but one global temp table is having null stats and so perhaps fo=
r
>>> that the dynamic sampling is triggered. In our case we are not using bi=
nd
>>> variables in this query but literals. But anyway , I am planning to set=
 the
>>> paralle_force_local to TRUE in the session level and along with that we
>>> will collect the stats on all the three tables setting all of them to u=
se
>>> private session level stats , hopefully that will also make optimizer g=
et
>>> rid of the dynamic sampling part.
>>>
>>> We can't set the dynamic sampling level to zero for this query because
>>> we are many times getting a bad execution path and to help fix that we =
need
>>> accurate stats , so I think private session level stats for all three
>>> tables is the one to go for in our case. Correct me if wrong.
>>>
>>> Also can you please share the exact bug which you are talking of.
>>>
>>>
>>> Note
>>>
>>> -----
>>>
>>>   - dynamic statistics used: dynamic sampling (level=3D2)
>>>
>>>
>>> Regards
>>>
>>> Pap
>>>
>>> On Tue, Feb 2, 2021 at 2:32 PM Mohamed Houri <mohamed.houri@gmail.com>
>>> wrote:
>>>
>>>>
>>>> I have faced a similar 12cR1 issue of parallel slave (PX) refusing to
>>>> share the execution plan of their query coordinator (QC) despite both =
QC
>>>> and its PX slaves were in the same instance. It turned to be a kind of=
 bug
>>>> because of the usage of bind variable (yes using bind variables !!)
>>>> and dynamic sampling.
>>>>
>>>>  But here you are using literals and GTT with private statistics. Bear
>>>> in mind that when you are using session private statistics Oracle will=
 not
>>>> propagate GTT private statistics from one session to another one. It
>>>> achieves this by invalidating similar cursor between sessions.
>>>>
>>>>
>>>> https://hourim.wordpress.com/2019/07/30/global-temporary-table-private=
-statistic
>>>> <https://hourim.wordpress.com/2019/07/30/global-temporary-table-privat=
e-statistics/>
>>>>
>>>>
>>>> That=E2=80=99s said I haven=E2=80=99t tested how Oracle will consider =
the QC
>>>> coordinator session per regards to its PX slave sessions in case of pr=
ivate
>>>> GTT sessions.
>>>>
>>>> You can see if your execution plan has the following Note signaling
>>>> that it has used a GTT private statistics or not
>>>>
>>>> Note
>>>>
>>>> -----
>>>>
>>>> -- Global temporary table session private statistics used
>>>>
>>>>
>>>>
>>>> And use Tanel Poder nonshared sql script to get the reason for which P=
X
>>>> slaves are refusing the share the execution plan of their QC
>>>>
>>>>
>>>> Best Regards
>>>>
>>>> Mohamed Houri
>>>>
>>>> Le mar. 2 f=C3=A9vr. 2021 =C3=A0 09:42, Chinar Aliyev <chinaraliyev@gm=
ail.com> a
>>>> =C3=A9crit :
>>>>
>>>>> Hi,
>>>>> There are several bugs in Oracle Support and might one of them be
>>>>> appropriate for your case.
>>>>>
>>>>> It happens when QC and PX Slaves are allocated in different instances=
,
>>>>> and a remote slave has to parse the statement based on the info sent =
by
>>>>> QC (to achieve the same execution plan generated by QC).
>>>>> To reduce parsing you can force it as Sayan has mentioned. Also, chec=
k
>>>>> support notes/bugs.
>>>>>
>>>>> Best Regards
>>>>>
>>>>> On Tue, Feb 2, 2021 at 11:12 AM Pap <oracle.developer35@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks a lot.
>>>>>>
>>>>>> We will definitely try rerunning by setting session level
>>>>>> "parallel_force_local" parameter to true as we set this value as 'FA=
LSE' in
>>>>>> v$parameter.
>>>>>>
>>>>>> *"parallel slaves will not be able to join the execution because the=
y
>>>>>> are not able to reproduce the same execution plan as the coordinator=
. "*
>>>>>>
>>>>>> I am a bit confused with above statement on the dynamic sampling
>>>>>> part, I see out of three global temporary tables used in this query =
, two
>>>>>> of them are using private session level stats as it's gathered insid=
e the
>>>>>> code. But one(RTNI) is having stats set as NULL and also its locked =
, which
>>>>>> means the dynamic sampling must have been triggered for that table o=
nly.
>>>>>> But in that case too, i am not able to understand how the dynamic sa=
mpling
>>>>>> can be the cause, can you please explain bit more. As because , my
>>>>>> understanding is parsing will happen at the first stage and till tha=
t time
>>>>>> parallel slaves wont get involve in real execution. So as the table =
related
>>>>>> info will be available in the node-3 as that being the session of th=
e query
>>>>>> coordinator/parent session , so that should only do the parsing work=
.
>>>>>> Please correct me if wrong.
>>>>>>
>>>>>> Regards
>>>>>> Pap
>>>>>>
>>>>>> On Tue, Feb 2, 2021 at 11:20 AM Laurentiu Oprea <
>>>>>> laurentiu.oprea06@gmail.com> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Additional to what has been mentioned, usually this issue is
>>>>>>> caused by the fact that parallel slaves will not be able to join th=
e
>>>>>>> execution because they are not able to reproduce the same execution=
 plan as
>>>>>>> the coordinator.
>>>>>>> Dynamic Sampling is often one of the root causes for this issue, If
>>>>>>> DS is not helping in your case you can create a spl patch for that =
sql with
>>>>>>> a lower level of DS or even disable it : dynamic_sampling(0).
>>>>>>>
>>>>>>> Good luck
>>>>>>>
>>>>>>> =C3=8En lun., 1 feb. 2021 la 23:53, Sayan Malakshinov <xt.and.r@gma=
il.com>
>>>>>>> a scris:
>>>>>>>
>>>>>>>> Hi Pap,
>>>>>>>>
>>>>>>>> Have you tried to set parallel_force_local=3Dtrue? Your QC is on N=
ode
>>>>>>>> 3 while your slaves are on Node 2:
>>>>>>>> 1. Parallel queries usually work much better if they don't need to
>>>>>>>> send data between nodes and fight for concurrent access to the sam=
e data;
>>>>>>>> 2. You are using global temporary tables, so their data is private
>>>>>>>> for your session and stored on the same node as your session. In c=
ase of
>>>>>>>> parallel access to GTT, QC has to send also their segment info, so=
 your
>>>>>>>> slaves have to request data from it, ie from Node 3.
>>>>>>>> 3. Also that means that node 2 have to parse your query too for
>>>>>>>> your slaves (sometimes it even leads to more child cursors)
>>>>>>>>
>>>>>>>> On Mon, Feb 1, 2021 at 10:28 PM Pap <oracle.developer35@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello All, We are seeing some odd behaviour. Its version
>>>>>>>>> 12.1.0.2.0 of oracle. And a small query(finishing in <1 minutes) =
which is
>>>>>>>>> executing in parallel(2) is experiencing "library cache lock" and=
 "cursor:
>>>>>>>>> pin S wait on X" between its own slaves. I mean to say the blocki=
ng session
>>>>>>>>> is appearing as its own slave sessions. We have "parallel_degree_=
policy"
>>>>>>>>> set as MANUAL in v$parametr. This query is running for different =
literals
>>>>>>>>> one after another multiple times in a loop fashion. And all these=
 samples
>>>>>>>>> logged in dba_hist_active_sess_history showing IN_PARSE as 'Y'. D=
ueto these
>>>>>>>>> waits the overall execution time of the process is going beyond ~=
5hrs+. The
>>>>>>>>> CPU and IO waits as noted in sql monitor is very small. Wondering=
 how
>>>>>>>>> parallel slave processes of the same query are blocking each othe=
r during
>>>>>>>>> parsing itself. Or are we hitting any bug in this version?
>>>>>>>>>
>>>>>>>>> Attached is the sql and its run time sql monitor. And all the
>>>>>>>>> tables used in this query are global temporary tables "on commit =
preserve
>>>>>>>>> row" types.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks And Regards
>>>>>>>>>
>>>>>>>>> Pap
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Sayan Malakshinov
>>>>>>>> Oracle performance tuning engineer
>>>>>>>> Oracle ACE Associate
>>>>>>>> http://orasql.org
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> *Chinar Aliyev*
>>>>>
>>>>>
>>>>> Visit My         :Blog <http://chinaraliyev.wordpress.com/>
>>>>> Let's Connect -
>>>>> <http://fr.linkedin.com/pub/mohamed-houri/11/329/857/>*Linkedin
>>>>> Profile <https://www.linkedin.com/in/chinaraliyev/>*
>>>>>
>>>>> My Twitter <https://twitter.com/MohamedHouri>      - ChinarAliyev
>>>>> <https://twitter.com/ChinarAliyev>
>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> Houri Mohamed
>>>>
>>>> Oracle DBA-Developer-Performance & Tuning
>>>>
>>>> Visit My         - Blog <http://www.hourim.wordpress.com/>
>>>>
>>>> Let's Connect -  <http://fr.linkedin.com/pub/mohamed-houri/11/329/857/=
>*Linkedin
>>>> Profile <http://fr.linkedin.com/pub/mohamed-houri/11/329/857/>*
>>>>
>>>> My Twitter <https://twitter.com/MohamedHouri>      - MohamedHouri
>>>> <https://twitter.com/MohamedHouri>
>>>>
>>>>
>
> --
>
> Houri Mohamed
>
> Oracle DBA-Developer-Performance & Tuning
>
> Visit My         - Blog <http://www.hourim.wordpress.com/>
>
> Let's Connect -  <http://fr.linkedin.com/pub/mohamed-houri/11/329/857/>*L=
inkedin
> Profile <http://fr.linkedin.com/pub/mohamed-houri/11/329/857/>*
>
> My Twitter <https://twitter.com/MohamedHouri>      - MohamedHouri
> <https://twitter.com/MohamedHouri>
>
>

--00000000000012223105ba57de0b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Nice one, thanks Mohamed. I`ll update on this thread once =
I reach the bottom of my issues as well.=C2=A0</div><br><div class=3D"gmail=
_quote"><div dir=3D"ltr" class=3D"gmail_attr">=C3=8En mar., 2 feb. 2021 la =
12:19, Mohamed Houri &lt;<a href=3D"mailto:mohamed.houri@gmail.com">mohamed=
.houri@gmail.com</a>&gt; a scris:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex"><div dir=3D"ltr"><p class=3D"MsoNormal" style=3D"margin:0=
cm 0cm 10pt;line-height:115%;font-size:11pt;font-family:Calibri,sans-serif"=
><span lang=3D"EN-US" style=3D"font-size:12pt;line-height:115%">@Laurentiu<=
/span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">As I mentioned above it was a kind of bug or k=
ind of =E2=80=9Cwe
don=E2=80=99t know why when using bind variable PX slaves were refusing to =
share the
execution plan of their QC=E2=80=9D. I have had an excellent private discus=
sion with
Oracle support engineer about this issue and he clearly explained that this=
 was
reported to the CBO team.</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">So, I am sorry I haven=E2=80=99t any bug to sh=
are with you for
this issue. Moreover, it happened in 12cR1 and, as I said above, I still ha=
ve
not encountered a similar issue in 12cR2, 18c, and 19c where I am using
parallelism extensively</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">@Pap</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">My answer was that, <b>probably</b>, your para=
llel parsing issue is due to the usage of GTT
private statistics. So if your planned fix tentative will reveal to be usel=
ess,
I would then suggest checking if using GTT shared statistics will get rid o=
f
the library cache lock and cursor pin S wait on x wait events. If so then y=
ou
have to balance between sharing inadequate GTT statistics and parsing effec=
ts.</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">=C2=A0</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><i><span lang=3D"EN-US" style=3D"f=
ont-size:9.5pt;line-height:115%;font-family:Verdana,sans-serif;color:black;=
background-image:initial;background-position:initial;background-size:initia=
l;background-repeat:initial;background-origin:initial;background-clip:initi=
al">=E2=80=9CAll things being equal
you must balance between the performance improvement brought by this new GT=
T
feature and the parsing side effect it introduces because of the underlying
cursor invalidation. In my client case,</span></i><i><span lang=3D"EN-US" s=
tyle=3D"font-size:9.5pt;line-height:115%;font-family:Verdana,sans-serif;col=
or:black"> <span style=3D"background-image:initial;background-position:init=
ial;background-size:initial;background-repeat:initial;background-origin:ini=
tial;background-clip:initial">the Library cache and Cursor Pin S wait on X =
wait
events introduced by the SESSION PRIVATE statistics largely outweigh the
performance penalty that comes when the 49 streams share the same GTT
statistics.</span>=E2=80=9D</span></i></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">=C2=A0</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">Best regards</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">Mohamed Houri</span></p></div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Le=C2=A0mar. 2 f=C3=
=A9vr. 2021 =C3=A0=C2=A011:10, Laurentiu Oprea &lt;<a href=3D"mailto:lauren=
tiu.oprea06@gmail.com" target=3D"_blank">laurentiu.oprea06@gmail.com</a>&gt=
; a =C3=A9crit=C2=A0:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex"><div dir=3D"ltr">Hello Pap,<div><br></div><div>I was mentioning DS in=
 the context of possible bug as well. If L2 is used most probably=C2=A0you =
don&#39;t have an issue around this and your issue might be around GTTs sta=
ts.</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gma=
il_attr">=C3=8En mar., 2 feb. 2021 la 11:57, Pap &lt;<a href=3D"mailto:orac=
le.developer35@gmail.com" target=3D"_blank">oracle.developer35@gmail.com</a=
>&gt; a scris:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0=
px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><=
div dir=3D"ltr"><div>Thanks much.</div><div><br></div><div>For that query ,=
 I see the below note section. And as i mentioned are using private session=
 level stats collected for two of the global temporary tables but one globa=
l temp table is having null stats and so perhaps for that the dynamic sampl=
ing is triggered.=20

In our case we are not using bind variables in this query but literals. But=
 anyway , I am planning to set the paralle_force_local to TRUE in the sessi=
on level and along with that we will collect the stats on all the three tab=
les setting all of them to use private session level stats , hopefully that=
 will also make optimizer=C2=A0get rid of the dynamic sampling part.=C2=A0<=
/div><div><br></div><div>We can&#39;t set the dynamic sampling level to zer=
o for this query because we are many times getting a bad execution path and=
 to help fix that we need accurate stats , so I think private session level=
 stats for all three tables is the one to go for in our case. Correct me if=
 wrong.</div><div><br></div><div>Also can you please share the exact bug wh=
ich you are talking of.=C2=A0</div><div><br></div><div><br></div><div><p st=
yle=3D"font-family:&quot;Oracle Sans&quot;;color:rgb(85,90,98);font-size:14=
px;word-break:break-word;line-height:inherit;box-sizing:border-box;padding:=
0px;margin:3px 0px 14px;border:0px;vertical-align:baseline;outline:0px;text=
-overflow:ellipsis;white-space:pre-wrap">Note</p><p style=3D"font-family:&q=
uot;Oracle Sans&quot;;color:rgb(85,90,98);font-size:14px;word-break:break-w=
ord;line-height:inherit;box-sizing:border-box;padding:0px;margin:3px 0px 14=
px;border:0px;vertical-align:baseline;outline:0px;text-overflow:ellipsis;wh=
ite-space:pre-wrap">-----</p><p style=3D"font-family:&quot;Oracle Sans&quot=
;;color:rgb(85,90,98);font-size:14px;word-break:break-word;line-height:inhe=
rit;box-sizing:border-box;padding:0px;margin:3px 0px;border:0px;vertical-al=
ign:baseline;outline:0px;text-overflow:ellipsis;white-space:pre-wrap">=C2=
=A0=C2=A0- dynamic statistics used: dynamic sampling (level=3D2)</p><p styl=
e=3D"font-family:&quot;Oracle Sans&quot;;color:rgb(85,90,98);font-size:14px=
;word-break:break-word;line-height:inherit;box-sizing:border-box;padding:0p=
x;margin:3px 0px;border:0px;vertical-align:baseline;outline:0px;text-overfl=
ow:ellipsis;white-space:pre-wrap"><br></p><p style=3D"font-family:&quot;Ora=
cle Sans&quot;;color:rgb(85,90,98);font-size:14px;word-break:break-word;lin=
e-height:inherit;box-sizing:border-box;padding:0px;margin:3px 0px;border:0p=
x;vertical-align:baseline;outline:0px;text-overflow:ellipsis;white-space:pr=
e-wrap">Regards</p><p style=3D"font-family:&quot;Oracle Sans&quot;;color:rg=
b(85,90,98);font-size:14px;word-break:break-word;line-height:inherit;box-si=
zing:border-box;padding:0px;margin:3px 0px;border:0px;vertical-align:baseli=
ne;outline:0px;text-overflow:ellipsis;white-space:pre-wrap">Pap</p></div></=
div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On=
 Tue, Feb 2, 2021 at 2:32 PM Mohamed Houri &lt;<a href=3D"mailto:mohamed.ho=
uri@gmail.com" target=3D"_blank">mohamed.houri@gmail.com</a>&gt; wrote:<br>=
</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b=
order-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><p=
 class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-siz=
e:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font-s=
ize:12pt;line-height:115%"><br></span></p><p class=3D"MsoNormal" style=3D"m=
argin:0cm 0cm 10pt;line-height:115%;font-size:11pt;font-family:Calibri,sans=
-serif"><span lang=3D"EN-US" style=3D"font-size:12pt;line-height:115%">I ha=
ve faced a similar 12cR1 issue of parallel slave (PX)
refusing to share the execution plan of their query coordinator (QC) despit=
e
both QC and its PX slaves were in the same instance. It turned to be a kind=
 of
bug because of the usage of bind variable (yes using bind variables !!</spa=
n><span lang=3D"EN-US" style=3D"font-size:12pt;line-height:115%">) and dyna=
mic
sampling. </span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">=C2=A0But here you
are using literals and GTT with private statistics. Bear in mind that when =
you
are using session private statistics Oracle will not propagate GTT private
statistics from one session to another one. It achieves this by invalidatin=
g
similar cursor between sessions.</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%"><a href=3D"https://hourim.wordpress.com/2019/0=
7/30/global-temporary-table-private-statistics/" style=3D"color:blue" targe=
t=3D"_blank">https://hourim.wordpress.com/2019/07/30/global-temporary-table=
-private-statistic</a></span></p><p class=3D"MsoNormal" style=3D"margin:0cm=
 0cm 10pt;line-height:115%;font-size:11pt;font-family:Calibri,sans-serif"><=
br></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">That=E2=80=99s said I haven=E2=80=99t tested h=
ow Oracle will consider
the QC coordinator session per regards to its PX slave sessions in case of
private GTT sessions. </span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">You can see if your execution plan has the fol=
lowing
Note signaling that it has used a GTT private statistics or not</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 0.0001pt;line-height:19.8pt;=
font-size:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=
=3D"font-size:12pt;font-family:&quot;Courier New&quot;;color:black">Note </=
span><span lang=3D"EN-US" style=3D"font-size:12pt;font-family:Consolas;colo=
r:black"></span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 0.0001pt;line-height:19.8pt;=
font-size:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=
=3D"font-size:12pt;font-family:&quot;Courier New&quot;;color:black">----- <=
/span><span lang=3D"EN-US" style=3D"font-size:12pt;font-family:Consolas;col=
or:black"></span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 0.0001pt;line-height:19.8pt;=
font-size:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=
=3D"font-size:12pt;font-family:&quot;Courier New&quot;;color:black">-- Glob=
al temporary table session private statistics
used</span><span lang=3D"EN-US" style=3D"font-size:12pt;font-family:Consola=
s;color:black"></span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">=C2=A0</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">And use Tanel Poder nonshared sql script to ge=
t the
reason for which PX slaves are refusing the share the execution plan of the=
ir
QC</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><br></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">Best Regards</span></p>

<p class=3D"MsoNormal" style=3D"margin:0cm 0cm 10pt;line-height:115%;font-s=
ize:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" style=3D"font=
-size:12pt;line-height:115%">Mohamed Houri</span></p></div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Le=C2=A0mar. 2 f=C3=
=A9vr. 2021 =C3=A0=C2=A009:42, Chinar Aliyev &lt;<a href=3D"mailto:chinaral=
iyev@gmail.com" target=3D"_blank">chinaraliyev@gmail.com</a>&gt; a =C3=A9cr=
it=C2=A0:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div d=
ir=3D"ltr"><div class=3D"gmail_default"><font face=3D"tahoma, sans-serif">H=
i,</font></div><div class=3D"gmail_default"><font face=3D"tahoma, sans-seri=
f">There are several bugs in Oracle Support and might one of them be approp=
riate for your case.</font></div><div class=3D"gmail_default"><font face=3D=
"tahoma, sans-serif"><br></font></div><div class=3D"gmail_default"><font fa=
ce=3D"tahoma, sans-serif">It happens=C2=A0when QC and PX Slaves are allocat=
ed in different instances, and a remote=C2=A0slave has to parse the stateme=
nt based on the  info=C2=A0</font>sent=C2=A0<span style=3D"font-family:taho=
ma,sans-serif">by QC (to achieve the same=C2=A0execution plan generated by =
QC).=C2=A0</span></div><div class=3D"gmail_default"><span style=3D"font-fam=
ily:tahoma,sans-serif">To reduce parsing you can force it as Sayan=C2=A0has=
 mentioned. Also, check support=C2=A0notes/bugs.</span></div><div class=3D"=
gmail_default"><span style=3D"font-family:tahoma,sans-serif"><br></span></d=
iv><div class=3D"gmail_default"><span style=3D"font-family:tahoma,sans-seri=
f">Best Regards=C2=A0</span></div></div><br><div class=3D"gmail_quote"><div=
 dir=3D"ltr" class=3D"gmail_attr">On Tue, Feb 2, 2021 at 11:12 AM Pap &lt;<=
a href=3D"mailto:oracle.developer35@gmail.com" target=3D"_blank">oracle.dev=
eloper35@gmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr">Thanks a lot.<div><br><=
/div><div>We will definitely try rerunning by setting session level &quot;p=
arallel_force_local&quot; parameter to true as we set this value as &#39;FA=
LSE&#39; in v$parameter.</div><div><br></div><div><i>&quot;parallel slaves =
will not be able to join the execution because they are not able to reprodu=
ce the same execution plan as the coordinator. &quot;</i></div><div><br></d=
iv><div>I am a bit confused with above statement on the dynamic=C2=A0sampli=
ng part, I see out of three global temporary tables used in this query , tw=
o of them are using private session level stats as it&#39;s gathered inside=
=C2=A0the code. But one(RTNI) is having stats set as NULL and also its lock=
ed , which means the dynamic sampling must have been triggered for that tab=
le only. But in that case too, i am not able to understand how the dynamic =
sampling can be the cause, can you please explain bit more. As because , my=
 understanding is parsing will happen at the first stage and till that time=
 parallel slaves wont get involve in real execution. So as the table relate=
d info will be available in the node-3 as that being the session of the que=
ry coordinator/parent session , so that should only do the parsing work. Pl=
ease correct me if wrong.</div><div><br></div><div>Regards</div><div>Pap</d=
iv></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gma=
il_attr">On Tue, Feb 2, 2021 at 11:20 AM Laurentiu Oprea &lt;<a href=3D"mai=
lto:laurentiu.oprea06@gmail.com" target=3D"_blank">laurentiu.oprea06@gmail.=
com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"marg=
in:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1e=
x"><div dir=3D"ltr">Hello,<div><br></div><div>Additional=C2=A0to what has b=
een mentioned, usually this issue is caused=C2=A0by the fact that parallel =
slaves will not be able to join the execution because they are not able to =
reproduce the same execution plan as the coordinator.=C2=A0</div><div>Dynam=
ic Sampling is often one of the root causes for this issue, If DS is not he=
lping in your case you can create a spl patch for that sql with a lower lev=
el of DS or even disable it : dynamic_sampling(0).</div><div><br></div><div=
>Good luck</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=
=3D"gmail_attr">=C3=8En lun., 1 feb. 2021 la 23:53, Sayan Malakshinov &lt;<=
a href=3D"mailto:xt.and.r@gmail.com" target=3D"_blank">xt.and.r@gmail.com</=
a>&gt; a scris:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:=
0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">=
<div dir=3D"ltr">Hi Pap,<div><br></div><div>Have you tried to set=C2=A0para=
llel_force_local=3Dtrue? Your QC is on Node 3 while your slaves are on Node=
 2:</div><div>1. Parallel queries usually work much better if they don&#39;=
t need to send data between nodes and fight for concurrent access to the sa=
me data;</div><div>2. You are using global temporary tables, so their data =
is private for your session and stored on the same node as your session. In=
 case of parallel access to GTT, QC has to send also their segment info, so=
 your slaves have to request data from it, ie from Node 3.</div><div>3. Als=
o that means that node 2 have to parse your query too for your slaves (some=
times it even leads to more child cursors)</div></div><br><div class=3D"gma=
il_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Feb 1, 2021 at 10:2=
8 PM Pap &lt;<a href=3D"mailto:oracle.developer35@gmail.com" target=3D"_bla=
nk">oracle.developer35@gmail.com</a>&gt; wrote:<br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex"><div dir=3D"ltr"><p style=3D"font-family:&=
quot;Oracle Sans&quot;;color:rgb(85,90,98);font-size:14px;word-break:break-=
word;line-height:inherit;box-sizing:border-box;padding:0px;margin:3px 0px 1=
4px;border:0px;vertical-align:baseline;outline:0px;text-overflow:ellipsis;w=
hite-space:pre-wrap"></p><p style=3D"font-family:&quot;Oracle Sans&quot;;co=
lor:rgb(85,90,98);font-size:14px;word-break:break-word;line-height:inherit;=
box-sizing:border-box;padding:0px;margin:3px 0px 14px;border:0px;vertical-a=
lign:baseline;outline:0px;text-overflow:ellipsis;white-space:pre-wrap">Hell=
o All, We are seeing some odd behaviour. Its version 12.1.0.2.0 of oracle. =
And a small query(finishing in &lt;1 minutes) which is executing in paralle=
l(2) is experiencing &quot;library cache lock&quot; and &quot;cursor: pin S=
 wait on X&quot; between its own slaves. I mean to say the blocking session=
 is appearing as its own slave sessions. We have &quot;parallel_degree_poli=
cy&quot; set as MANUAL in v$parametr. This query is running for different l=
iterals one after another multiple times in a loop fashion. And all these s=
amples logged in dba_hist_active_sess_history showing IN_PARSE as &#39;Y&#3=
9;. Dueto these waits the overall execution time of the process is going be=
yond ~5hrs+. The CPU and IO waits as noted in sql monitor is very small. Wo=
ndering how parallel slave processes of the same query are blocking each ot=
her during parsing itself. Or are we hitting any bug in this version?<br></=
p><p style=3D"font-family:&quot;Oracle Sans&quot;;color:rgb(85,90,98);font-=
size:14px;word-break:break-word;line-height:inherit;box-sizing:border-box;p=
adding:0px;margin:3px 0px 14px;border:0px;vertical-align:baseline;outline:0=
px;text-overflow:ellipsis;white-space:pre-wrap">Attached is the sql and its=
 run time sql monitor. And all the tables used in this query are global tem=
porary tables &quot;on commit preserve row&quot; types.<br></p><p style=3D"=
font-family:&quot;Oracle Sans&quot;;color:rgb(85,90,98);font-size:14px;word=
-break:break-word;line-height:inherit;box-sizing:border-box;padding:0px;mar=
gin:3px 0px 14px;border:0px;vertical-align:baseline;outline:0px;text-overfl=
ow:ellipsis;white-space:pre-wrap"><br></p><p style=3D"font-family:&quot;Ora=
cle Sans&quot;;color:rgb(85,90,98);font-size:14px;word-break:break-word;lin=
e-height:inherit;box-sizing:border-box;padding:0px;margin:3px 0px 14px;bord=
er:0px;vertical-align:baseline;outline:0px;text-overflow:ellipsis;white-spa=
ce:pre-wrap"><br></p><p style=3D"font-family:&quot;Oracle Sans&quot;;color:=
rgb(85,90,98);font-size:14px;word-break:break-word;line-height:inherit;box-=
sizing:border-box;padding:0px;margin:3px 0px 14px;border:0px;vertical-align=
:baseline;outline:0px;text-overflow:ellipsis;white-space:pre-wrap">Thanks A=
nd Regards</p><p style=3D"font-family:&quot;Oracle Sans&quot;;color:rgb(85,=
90,98);font-size:14px;word-break:break-word;line-height:inherit;box-sizing:=
border-box;padding:0px;margin:3px 0px 14px;border:0px;vertical-align:baseli=
ne;outline:0px;text-overflow:ellipsis;white-space:pre-wrap">Pap</p></div>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
><div dir=3D"ltr"><div><div dir=3D"ltr"><div>Best regards,<br>Sayan Malaksh=
inov</div><span style=3D"font-size:13.6px">Oracle performance tuning engine=
er</span><br style=3D"font-size:13.6px"><div>Oracle ACE Associate<br><a hre=
f=3D"http://orasql.org" target=3D"_blank">http://orasql.org</a></div></div>=
</div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=
=3D"ltr"><div><div dir=3D"ltr"><div><b style=3D"color:rgb(0,0,0);font-famil=
y:&quot;trebuchet ms&quot;,sans-serif">Chinar Aliyev</b></div><div><br></di=
v><div><img src=3D"https://docs.google.com/uc?export=3Ddownload&amp;id=3D1H=
Yf9FoBNXnnwwf-tYZFjAkMCrzg_Yddo&amp;revid=3D0B-QXE3D0UMX1ZFN1d3RhcXZCODhjU3=
JEVkJsa1F2eEp0eDRVPQ"><br></div><div><span style=3D"font-size:11pt;font-fam=
ily:&quot;Times New Roman&quot;,serif;color:rgb(33,33,33)">Visit My=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 :<a href=3D"http://chinaraliyev.=
wordpress.com/" target=3D"_blank">Blog</a></span></div><div><span style=3D"=
font-family:Arial,sans-serif;font-size:10pt">Let&#39;s Connect -</span><a h=
ref=3D"http://fr.linkedin.com/pub/mohamed-houri/11/329/857/" style=3D"font-=
family:Arial,sans-serif;font-size:10pt;color:rgb(17,85,204)" target=3D"_bla=
nk">=C2=A0</a><u style=3D"font-family:Arial,sans-serif;font-size:10pt"><a h=
ref=3D"https://www.linkedin.com/in/chinaraliyev/" style=3D"color:rgb(17,85,=
204)" target=3D"_blank">Linkedin Profile</a></u><br></div><div><p style=3D"=
margin:0cm 0cm 0.0001pt"></p><p style=3D"margin:0cm 0cm 0.0001pt"><span sty=
le=3D"font-size:10pt;font-family:Arial,sans-serif">My=C2=A0<a href=3D"https=
://twitter.com/MohamedHouri" style=3D"color:rgb(17,85,204)" target=3D"_blan=
k"><span style=3D"color:windowtext">Twitter</span></a>=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 -=C2=A0<a href=3D"https://twitter.com/ChinarAliyev" style=3D"colo=
r:rgb(17,85,204)" target=3D"_blank">ChinarAliyev</a></span></p></div><div><=
br></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
><div dir=3D"ltr"><div dir=3D"ltr"><p style=3D"margin:0cm 0cm 0.0001pt"><sp=
an style=3D"font-size:10pt;font-family:Arial,sans-serif">Houri Mohamed=C2=
=A0</span></p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size=
:10pt;font-family:Arial,sans-serif">Oracle
DBA-Developer-Performance &amp; Tuning=C2=A0</span></p><p style=3D"margin:0=
cm 0cm 0.0001pt">



</p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size:10pt;font=
-family:Arial,sans-serif">Visit My =C2=A0 =C2=A0 =C2=A0
=C2=A0 -=C2=A0<a href=3D"http://www.hourim.wordpress.com/" target=3D"_blank=
">Blog</a></span></p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"fo=
nt-size:10pt;font-family:Arial,sans-serif">Let&#39;s
Connect -<a href=3D"http://fr.linkedin.com/pub/mohamed-houri/11/329/857/" t=
arget=3D"_blank">=C2=A0</a><u><a href=3D"http://fr.linkedin.com/pub/mohamed=
-houri/11/329/857/" target=3D"_blank">Linkedin
Profile</a></u></span></p><p style=3D"margin:0cm 0cm 0.0001pt">



</p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size:10pt;font=
-family:Arial,sans-serif">My=C2=A0<a href=3D"https://twitter.com/MohamedHou=
ri" target=3D"_blank"><span style=3D"color:windowtext">Twitter</span></a>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -<span>=C2=A0</span><a href=3D"https://twitt=
er.com/MohamedHouri" target=3D"_blank">MohamedHouri</a></span></p><p style=
=3D"margin-bottom:0.0001pt"><span style=3D"font-size:10pt;font-family:Arial=
,sans-serif"></span></p></div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
><div dir=3D"ltr"><div dir=3D"ltr"><p style=3D"margin:0cm 0cm 0.0001pt"><sp=
an style=3D"font-size:10pt;font-family:Arial,sans-serif">Houri Mohamed=C2=
=A0</span></p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size=
:10pt;font-family:Arial,sans-serif">Oracle
DBA-Developer-Performance &amp; Tuning=C2=A0</span></p><p style=3D"margin:0=
cm 0cm 0.0001pt">



</p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size:10pt;font=
-family:Arial,sans-serif">Visit My =C2=A0 =C2=A0 =C2=A0
=C2=A0 -=C2=A0<a href=3D"http://www.hourim.wordpress.com/" target=3D"_blank=
">Blog</a></span></p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"fo=
nt-size:10pt;font-family:Arial,sans-serif">Let&#39;s
Connect -<a href=3D"http://fr.linkedin.com/pub/mohamed-houri/11/329/857/" t=
arget=3D"_blank">=C2=A0</a><u><a href=3D"http://fr.linkedin.com/pub/mohamed=
-houri/11/329/857/" target=3D"_blank">Linkedin
Profile</a></u></span></p><p style=3D"margin:0cm 0cm 0.0001pt">



</p><p style=3D"margin:0cm 0cm 0.0001pt"><span style=3D"font-size:10pt;font=
-family:Arial,sans-serif">My=C2=A0<a href=3D"https://twitter.com/MohamedHou=
ri" target=3D"_blank"><span style=3D"color:windowtext">Twitter</span></a>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -<span>=C2=A0</span><a href=3D"https://twitt=
er.com/MohamedHouri" target=3D"_blank">MohamedHouri</a></span></p><p style=
=3D"margin-bottom:0.0001pt"><span style=3D"font-size:10pt;font-family:Arial=
,sans-serif"></span></p></div></div></div>
</blockquote></div>

--00000000000012223105ba57de0b--

--
http://www.freelists.org/webpage/oracle-l



