Re: RAC install on Linux

From: Jared Still <jkstill_at_gmail.com>
Date: Fri, 23 Feb 2018 14:00:12 -0700
Message-ID: <CAORjz=OrxKW5yMi=FGNUmVxnpOcsjgb-8JqRQQjREusSgRYh4Q_at_mail.gmail.com>



"This behavior is a direct result of the instance parameter setting *'pre_page_sga
= true'*, which causes oracle processes to touch every single page in the SGA during process initialization. When not using huge pages the use of this parameter puts a lot of strain on the memory management of the OS when using large SGAs. "

This happens when huge pages are not used.

That is because of the large number of buffers required with 4K pagesize.

Probably not going to happen with a 2M pagesize unless the SGA is incredibly large.

I don't have time to do the math right now. :)

Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist Principal Consultant at Pythian
Pythian Blog http://www.pythian.com/blog/author/still/ Github: https://github.com/jkstill

On Thu, Feb 22, 2018 at 1:37 PM, Ruel, Chris <Chris.Ruel_at_lfg.com> wrote:

> Someone correct me if I am wrong but HP's do not lock memory. They just
> use a larger page size. In fact some people had problems locking memory
> with HP's enabled. I don't know if Oracle ever admitted to this being a
> bug. This note is a little older but I know it affected us as late as
> 11.2.0.4:
>
> Oracle Support Document 1276966.1 (Rac Nodes Frequently Evicted
> (Rebooted) By CRS When PRE_PAGE_SGA is Set to True) can be found at:
> https://support.oracle.com/epmos/faces/DocumentDisplay?id=1276966.1
>
> Also, looking at the thread below, this is news to me that you must use
> ASMM with HP's. We use HP's with standard memory management in some cases
> and it seems to work fine. I know that AMM is a no-no.
>
> Chris..
>
> _____________________________________________________________________
> Chris Ruel * Oracle Database Administrator * Lincoln Financial Group
> cruel_at_lfg.com * Desk:317.759.2172 * Cell 317.523.8482
>
> -----Original Message-----
> From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org]
> On Behalf Of Hameed, Amir
> Sent: Thursday, February 22, 2018 3:06 PM
> To: contact_at_soocs.de; neil_chandler_at_hotmail.com
> Cc: Oracle-L Freelists <oracle-l_at_freelists.org>
> Subject: RE: RAC install on Linux
>
> ***This email is from an external source. Only open links and attachments
> from a Trusted Sender.***
>
>
> In those VM shops where memory is over-subscribed, how do Huge Pages work
> in terms of locking memory pages so that they cannot be swapped?
>
> Thanks
> -----Original Message-----
> From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org]
> On Behalf Of Stefan Koehler
> Sent: Thursday, February 22, 2018 5:21 AM
> To: neil_chandler_at_hotmail.com
> Cc: Oracle-L Freelists <oracle-l_at_freelists.org>
> Subject: Re: RAC install on Linux
>
> Hello Neil,
>
> > You also have contiguous memory allocation and it can’t be swapped.
>
> Hugepages can also be non-contiguous aka. fragmented - called external
> fragmentation (https://lwn.net/Articles/376606/). AFAIK this is also the
> reason why Oracle has implemented enhancement request #14500387 and you get
> an "ORA-27107: AUTO value for USE_LARGE_PAGES parameter is no longer
> supported" (https://docs.oracle.com/database/121/ERRMG/ORA-24280.
> htm#GUID-7B2814DB-F946-4EB1-80B5-219B256E90C0__GUID-
> E1254AED-AB39-4D52-97E2-5D6011E161E7) nowadays for dynamic huge pages
> allocation via oradism.
>
> Best Regards
> Stefan Koehler
>
> Independent Oracle performance consultant and researcher
> Website: http://www.soocs.de
> Twitter: _at_OracleSK
>
> > Neil Chandler <neil_chandler_at_hotmail.com> hat am 22. Februar 2018 um
> 09:47 geschrieben:
> >
> > Transparent Hugepages should be disabled, to prevent the Kernel
> “helping you out” and causing CPU spikes and unpredictable performance hits.
> >
> > You should always be using Hugepages.
> >
> > They give a minor performance improvement and a significant memory
> saving in terms of the amount of memory needed to handle the pages - less
> Transaction Lookaside Buffers, which also means less TLB misses (which are
> expensive).
> >
> > You are handling the memory chopped up into 2MB pieces instead of 4K.
> But you also have a single shared memory TLB for Hugepages.
> >
> > The kernel has less work to do, bookkeeping fewer pointers in the TLB.
> >
> > You also have contiguous memory allocation and it can’t be swapped.
> >
> > If you are having problems with Hugepages, you have probably
> overallocated them (I’ve seen this several times at clients so it’s not
> uncommon). Hugepages can *only* by used for your SGA’s. All of your SGA’s
> should fit into the Hugepages and that should generally be no more than
> about 60% of the total server memory (but there are exceptions), leaving
> plenty of “normal” memory (small pages) for PGA , O/S and other stuff like
> monitoring agendas.
> >
> > As an added bonus, AMM can’t use Hugepages, so your are forced to use
> ASMM. AMM doesn’t work well and has been kind-of deprecated by oracle
> anyway - dbca won’t let you setup AMM if the server has more than 4GB of
> memory.
> >
> > Neil.
> > sent from my phone
> --
> http://www.freelists.org/webpage/oracle-l
>
>
> !�� � 0~���+-����
> ��� ���rW�
> Notice of Confidentiality: **This E-mail and any of its attachments may
> contain
> Lincoln National Corporation proprietary information, which is privileged,
> confidential,
> or subject to copyright belonging to the Lincoln National Corporation
> family of
> companies. This E-mail is intended solely for the use of the individual or
> entity to
> which it is addressed. If you are not the intended recipient of this
> E-mail, you are
> hereby notified that any dissemination, distribution, copying, or action
> taken in
> relation to the contents of and attachments to this E-mail is strictly
> prohibited
> and may be unlawful. If you have received this E-mail in error, please
> notify the
> sender immediately and permanently delete the original and any copy of
> this E-mail
> and any printout. Thank You.**
>

--
http://www.freelists.org/webpage/oracle-l
Received on Fri Feb 23 2018 - 22:00:12 CET

Original text of this message