Re: Solaris 10 and Oracle 10g - swap space problem
Date: Mon, 03 Aug 2009 09:21:02 -0500
Message-id: <4A76F24E.5050603_at_Sun.COM>
Hi Riyaj,
I think the issue here is with understanding what the data from "swap -s" means.
Here's from one of my internal systems:
# swap -l
swapfile dev swaplo blocks free /dev/dsk/c0t1d0s1 32,9 16 8201840 8201840 /dev/md/dsk/d209 85,209 16 67108848 67108848 /dev/md/dsk/d210 85,210 16 67108848 67108848 /dev/md/dsk/d211 85,211 16 67108848 67108848 /dev/md/dsk/d212 85,212 16 67108848 67108848 /dev/md/dsk/d213 85,213 16 67108848 67108848
# sys
XXX 08/03/09 10:02:16 SunOS 5.10 Generic_139555-04 sun4u
2520/0 MHz 8 CPUs 32.00 GB memory SUNW,SPARC-Enterprise
# swap -s
total: 71280k bytes allocated + 12368k reserved = 83648k used, 200001416k available
So the 200001416k is equal to the 32 * 5 + 4 GB in actual swap files plus 32 GB in memory minus the kernel space (and whatever is in /tmp; I didn't check).
When I start oracle here:
Total System Global Area 4932501504 bytes
Fixed Size 2047312 bytes Variable Size 620757680 bytes Database Buffers 4294967296 bytes Redo Buffers 14729216 bytesDatabase mounted.
Database opened.
# pmap -xs 9020 | grep -i ism
0000000380000000 4816896 4816896 - 4816896 4M rwxsR [ ism
shmid=0x1 ]
00000004A6000000 16 16 - 16 8K rwxsR [ ism
shmid=0x1 ]
So it's using ISM.
bur834-98# swap -s
total: 5140592k bytes allocated + 33968k reserved = 5174560k used, 194842040k
available
So now the available space is about 6G less, which is pretty much the total of the ~5G SGA plus swap space for the oracle processes.
If I set "sga_max_size = 7159641169" and restart:
Total System Global Area 7163871232 bytes
Fixed Size 2051616 bytes Variable Size 2852131296 bytes Database Buffers 4294967296 bytes Redo Buffers 14721024 bytesDatabase mounted.
Database opened.
# ps -ef | grep lgwr
oracle 11586 1 0 10:10:47 ? 0:00 ora_lgwr_XXX
bur834-98# pmap -xs 11586 | grep -i dism
0000000380000000 16384 16384 - - 4M rwxs- [
dism shmid=0x3 ]
0000000381000000 2191360 12288 - - - rwxs- [
dism shmid=0x3 ]
...
So it's now using DISM instead.
# swap -s
total: 5144680k bytes allocated + 2212928k reserved = 7357608k used, 187819608k
available
Now the total is reduced by an additional 7G to save space for the DISM backing. Note the other memory use hasn't changed, so the size in memory remains the same, about 5G. There's just extra swap space allocated.
So things are working as expected; it's just the interpretation of the "swap -s" that's causing problems.
Regards,
Dave Miller
Riyaj Shamsudeen wrote, On 08/01/09 12:01:
> Tanel
> I must disagree with you. It looks like swap is allocated even for ISM
> pages in Solaris. Please feel free to correct me if there is any
> misunderstanding from my part.
>
> Here is a small test case:
>
> RS: Intially, 7.4G of swap used
> swap -s
> total: 7207256k bytes allocated + 233128k reserved = 7440384k used,
> 40446656k available
>
> RS: ISM in use and SGA is locked as expected.
> Address Kbytes RSS Anon Locked Mode
> Mapped File
> ..
> 0000000380000000 6422544 6422544 - 6422544 rwxsR [
> ism shmid=0x300001c ]
>
> RS: No sga_max_size set in spfile either.
>
> strings spfilePROD.ora |grep -i sga
> *.sga_target=6272M
>
> RS: Shutting down...
> sqlplus "/ as sysdba"
>
> Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
> With the Partitioning, Real Application Clusters, OLAP, Data Mining
> and Real Application Testing options
>
> SQL> shutdown immediate;
> Database closed.
> Database dismounted.
> ORACLE instance shut down.
> SQL>
>
> RS: Just 500M of swap used after instance shutdown..
> wsqfinc1e> swap -s
> total: 451224k bytes allocated + 63928k reserved = 515152k used,
> 47400568k available
>
> RS: Restarting instance..
> ORACLE instance started.
>
> Total System Global Area 6576668672 bytes
> Fixed Size 2050464 bytes
> Variable Size 2583696992 bytes
> Database Buffers 3976200192 bytes
> Redo Buffers 14721024 bytes
> Database mounted.
> Database opened.
>
> RS: Swap is back to 7.32GB
> swap -s
> total: 7135384k bytes allocated + 191968k reserved = 7327352k used,
> 40566696k available
>
> RS: Still ISM..
> Address Kbytes RSS Anon Locked Mode
> Mapped File
> ...
> 0000000380000000 6422544 6422544 - 6422544 rwxsR [
> ism shmid=0x3000048 ]
>
> Even though, ISM is in use and sga_max_size is not in use, swap is
> allocated.
>
>
> Cheers
>
> Riyaj Shamsudeen
> Principal DBA,
> Ora!nternals - http://www.orainternals.com
> Specialists in Performance, Recovery and EBS11i
> Blog: http://orainternals.wordpress.com
>
>
> On Sat, Aug 1, 2009 at 12:51 AM, Tanel Poder <tanel_at_poderc.com
> <mailto:tanel_at_poderc.com>> wrote:
>
> If you use ISM (when SGA_MAX_SIZE = SGA_TARGET or when SGA_MAX_SIZE
> is unset in parameter file) then all SGA pages are locked in
> physical memory (by OS VM design) - thus no swap space needs to be
> reserved as the pages in memory aren't pageable.
>
> If you are using DISM (when SGA_MAX_SIZE > SGA_TARGET) then SGA is
> using pageable large pages in Solaris (which are individually locked
> in physical memory by Oracle's mlock requests). But as the pages are
> essentially pageable, swap space needs to be reserved when SGA is
> created.
>
> If you want to avoid your SGA causing the swap space reservation,
> unset your SGA_MAX_SIZE parameter.
>
> I have mentioned this on this blog entry (and in the comments
> section there's a link to a Solaris kernel internals presentation):
>
> http://blog.tanelpoder.com/2007/08/28/operating-systems-are-lazy-allocating-memory/
>
> --
> Tanel Poder
> http://blog.tanelpoder.com
>
>
>
> On Wed, Jul 29, 2009 at 7:31 AM, Crisler, Jon <Jon.Crisler_at_usi.com
> <mailto:Jon.Crisler_at_usi.com>> wrote:
>
> On Solaris 10, when I start my databases, I see my swap space
> usage increase in a one for one ratio with memory utilization.
> System has 64gb of memory. If I start up a 4gb database, I see
> 4gb of real memory used and 4gb of swap space used up as well
> (as shown by top, swap –l, swap –s etc.). This is purely a
> Oracle database server, and /etc/system and projects / resource
> controls seem to be set up, so I do not understand why swap is
> being touched. If I stop all oracle processes, swap drops down
> to about 100 Mb used.
>
>
>
> I have taken great pains to make sure that NO swap is being
> used, yet any little allocation of Oracle gets tossed into
> swap. I don’t know where to look after this- I would have
> suspected /etc/system or resource control / projects to be at
> fault, or /etc/security (ulimits) but everything looks ok.
> The system has 64g real, and about 54 gb swap- if I get close
> to 54gb memory allocated oracle will return out of memory
> errors.. Needless to say the system crawls.
>
>
>
> What am I overlooking ? TOP and swap –s / swap –l agree so its
> not a TOP anomaly
>
>
>
> Example from TOP – I have 20g free, so why so much used in swap ?
>
>
>
> load averages: 0.64, 0.57,
> 0.46
> 00:26:34
>
> 1073 processes:1071 sleeping, 2 on cpu
>
> CPU states: 98.3% idle, 1.2% user, 0.5% kernel, 0.0% iowait,
> 0.0% swap
>
> Memory: 64G real, 20G free, 40G swap in use, 17G swap free
>
>
>
> PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU
> COMMAND
>
> 1997 oracle 1 39 0 4376K 2592K cpu/50 0:02 0.04% top
>
> 3454 oracle 1 52 0 16G 16G cpu/32 0:00 0.03% oracle
>
> 3456 oracle 1 24 0 16G 16G sleep 0:00 0.03%
> oracle
>
> 18589 oracle 1 59 0 16G 16G sleep 0:13 0.02% oracle
>
>
>
> Swap –s
>
> total: 39170184k bytes allocated + 2262264k reserved = 41432448k
> used, 18017608k available
>
>
>
>
> --
> Tanel Poder
> http://blog.tanelpoder.com
>
>
-- http://www.freelists.org/webpage/oracle-lReceived on Mon Aug 03 2009 - 09:21:02 CDT