| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
 |
 | |||
| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
|
| |||
Home -> Community -> Mailing Lists -> Oracle-L -> RE: Limiting CPUs on SUN
Still hoping to find someone with SUN experience on this.
Below is part of the doc we had for the TRU64 change with used the "pset" command
Originally this was done by another DBA team on a server with 16 CPUs and 8.1.7.2.1
Thanks Larry Wolfson
Processor Sets
Compaq provides the capability with a number of their servers to define
groups of processors called processor sets or psets. Each pset can consist
of 1 to many processors. If no sets are defined, then all processes will
run in pset 0, which consists of all processors on the server. Through
commands such as "runon" and "pset_assign_pid", processes can be run on
specific psets.
Recommendation: This strategy was the only viable option and was chosen for use in production. The remainder of this article discusses how this strategy was implemented.
Implementation of Processor Sets
To properly implement processor sets with Oracle, the server and Oracle
first needed to be configured to work with them. Next, methods needed to be
determined on how to monitor the configuration. Lastly, there are different
strategies to implement processor sets, so the different strategies needed
to be tested.
Pset Configuration
The GS160 server has 4 separate QBBs (Quad Building Blocks), each holding 4
processors and 4GB of memory. The fastest memory access by a process occurs
when the processor it's running on and the memory the process is using are
in the same QBB. This is what is called "local access".
To support this type of access with Oracle, the kernel parameters rad_gh_regions[*] should be set, one per QBB. These kernel parameters define a local region of memory within the associated QBB to hold a slice of the SGA. The ideal configuration with Oracle is to divide the maximum SGA size by the number of QBBs, then set this value on each rad_gh_region[*] parameter. For the database involved, the SGA should never grow past 1GB, but to be safe a size of 1GB was chosen for each rad_gh_region[*] parameter.
Oracle, SAS, and Orchestrate all need to run on the server and are licensed for 8, 4, and 4 processors respectively. Therefore 3 psets were created to support this licensed software. To keep access local within each QBB whenever possible, the processors were assigned to psets relative to their QBB. Processors 0-3 were on QBB 0, processors 4-7 on QBB 1, processors 8-11 on QBB 2, and processors 12-15 on QBB 3 (see Appendix A for methods to map processors to QBBs). Therefore, pset 0 was created using processors 0-3, pset 1 held processors 4-7, and pset 3 held processors 8-15.
Since Oracle was relegated to processors 8-15, QBBs 2 and 3, the kernel parameters rad_gh_regions[2] and rad_gh_regions[3] were defined to 1024 (1GB). ssm_threshold was also set to value of 0, which appears to keep each SGA slice local within the rad_gh_regions slice.
Implementation
To force a process to run on a specific pset, you have two options, either
1), precede the execution of all Unix-level commands with "runon" (to run on
a pset or processor), or 2), assign Unix PIDs to a pset using
"pset_assign_pid". While both methods achieve the same goal, it was found
that assigning Unix PIDs to a pset was a better route. The main reason for
this conclusion was that the "runon" command causes the given command to run
in a separate shell, doubling the number of Unix processes. As the number
of concurrent processes grows, the "runon" method will use more and more
memory because of the extra Unix processes.
The assignment of Unix processes by PID to a pset was accomplished through the use of a script called "assign_pset.ksh" (see Appendix A for the code). This script calls "pset_assign_pid" to assign a Unix PID to a pset, based on the Unix username. By default the script will only reassign the parent PID (or PID of the process that called this script), with the assumption the code is sourced (run in the same shell by using ". assign_pset.ksh"). The script can also be run in (A)ll mode, which causes the script to check the pset of all processes running on the server.
"assign_pset.ksh" was added to the end of "/usr/local/XXX.profile", so that most Unix processes will get properly assigned to a pset during process creation. "assign_pset.ksh" was also added to "cron" under root and run in (A)ll mode every hour, to catch any Unix processes that did not run the standard process creation scripts, such as through "cron" or "su" without the dash.
Fortunately in Tru64, by default child processes run on the same pset as the parent. In this way all Oracle processes, background and server, run on the pset of the parent, which gets properly assigned during process creation.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org
[mailto:oracle-l-bounce_at_freelists.org]On Behalf Of Wolfson Larry -
lwolfs
Sent: Wednesday, May 12, 2004 6:37 PM
To: 'oracle-l_at_freelists.org'
Subject: RE: Limiting CPUs on SUN
I found these on GOOGLE re psrset:
http://www.sas.com/partners/directory/sun/sunfire.pdf
http://www.sun.com/sun-on-net/itworld/UIR980501perf.html
Anybody using this with Oracle?
Thanks Larry
-----Original Message-----
From: oracle-l-bounce_at_freelists.org
[mailto:oracle-l-bounce_at_freelists.org]On Behalf Of Wolfson Larry -
lwolfs
Sent: Wednesday, May 12, 2004 12:37 PM
To: oracle-l_at_freelists.org
Subject: Limiting CPUs on SUN
Wanted to check on this.
Client (Currently 8174) discovered they're only licensed for X CPUs where they have X+14. They want to limit this to be in compliance and still run associated apps.
Someone's looking at CPU_COUNT which I don't think really works at limiting anything. I've seen it set higher than actual by some DBAs, but didn't hear what benefit that really was.
Sun does have a psrset command I just heard of. Will that do the job for them?
Thanks
Larry Wolfson
We did this on TRU64 but we had to modify the OS ourselves. Not a popular decision.
NAME
psrset - creation and management of processor sets
SYNOPSIS psrset -a processor_set_id processor_id...
psrset -b processor_set_id pid...
psrset -c [processor_id]...
psrset -d processor_set_id
psrset -e processor_set_id command [argument(s)]
psrset -f processor_set_id
psrset [-i] [processor_set_id]...
psrset -n processor_set_id
psrset -p [processor_id]...
psrset -q [pid]...
psrset -r processor_id...
psrset -u pid...
DESCRIPTION The psrset utility controls the management of processor sets. Processor sets allow the binding of processes to groups of processors, rather than just a single processor. There are two types of processor sets, those created by the user using the psrset command or the pset_create(2) <@FPI2SOI;targetdocent=-%2F%2FSun%3A%3ASunSoft%2F%2FDOCUMENT+REFMAN2+Version +2.0%2F%2FEN;localinfo=pset-create-2;type=> system call, and those automatically created by the system. Processors assigned to user-created processor sets will run only LWPs that have been bound to that processor set, but system processor sets may run other LWPs as well.
System-created processor sets will not always exist on a given machine. When they exist, they will generally represent particular characteristics of the underlying machine, such as groups of processors that can communicate more quickly with each other than with other processors in the system. These processor sets cannot be modified or removed, but processes may be bound to them.
l
-- Archives are at http://www.freelists.org/archives/oracle-l/ FAQ is at http://www.freelists.org/help/fom-serve/cache/1.htmlReceived on Thu May 13 2004 - 12:03:40 CDT
-----------------------------------------------------------------
----------------------------------------------------------------
Please see the official ORACLE-L FAQ: http://www.orafaq.com
----------------------------------------------------------------
To unsubscribe send email to: oracle-l-request_at_freelists.org put 'unsubscribe' in the subject line. -- Archives are at http://www.freelists.org/archives/oracle-l/ FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
-----------------------------------------------------------------
----------------------------------------------------------------
Please see the official ORACLE-L FAQ: http://www.orafaq.com
----------------------------------------------------------------
To unsubscribe send email to: oracle-l-request_at_freelists.org put 'unsubscribe' in the subject line. -- Archives are at http://www.freelists.org/archives/oracle-l/ FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
-----------------------------------------------------------------
 |
 |