RE: df shows 100% on a empty filesystem-
Date: Thu, 17 Jan 2008 13:06:47 +0800
You need to kill ARCx processes if you want to stop them. Oracle instance will just restart the archivers if it finds them dead. Make sure you test this on your version/platform first though before using in live.
One reason which can leave open deleted files lying around can be faulty archivelog backup/transfer/compress scripts, which do not check whether the archivelog is still open before deleting/moving/compressing the file. Such scripts should always check whether the file is closed (and archiving is complete) either with fuser or by querying v$archived_log. Otherwise you may end in worse trouble than just filesystem full - having only partial copies of archivelogs.. and being unable to perform recovery.
-- Regards, Tanel Poder http://blog.tanelpoder.com <http://blog.tanelpoder.com/> _____ From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Prasad Sent: Thursday, January 17, 2008 09:55 To: krish.hariharan_at_quasardb.com Cc: Mir M. Mirhashimali; oracle-l Subject: Re: df shows 100% on a empty filesystem- Hi Krish, Even though alert log shows ARCHIVE LOG STOP has been deprecated I still tried alter system ARCHIVE LOG STOP for an hour ( as I have enabled the log_archive_dest_state_3 pointing to a different filesystem) but it didnt help me .. I do see lsof pointing to /ofs/archivelog01 but the problem is that there are multiple unix pid owned by oracle doing something on /ofs/archivelogs01 . Thanks -Prasad On Jan 16, 2008 5:05 PM, <krish.hariharan_at_quasardb.com> wrote: We have seen this in our large environments where several hundred gb gets tied up. Removing the files will not work since you are only manipulating the directory entries and unless the inode is released the OS will not release the space and that will happen only when the process that has those file handles open, closes them or is terminated. We end up shutting down the database and on occasion reboot the server. When you move files across file system you are again copying the files and then removing the directory entries in the source file systems. The process that has the files open is not affected by the rename/removal of a file's directory entry. I have looked at these processes using the files by using fuser but I haven't found that generally useful since it tends to dump all the oracle processes. The other utility that I know of but haven't used is lsof which gives the list of open files. A long shot: One approach you may be able to take is to issue an archive log stop for a brief period (not sure if this will stop the archiver or not since the alert log says it is deprecated and the processes are still hanging around) and see if it posts/causes the archiver to release the file handles and allow the os the release the space (assuming that you have sufficiently large and a large number of redo log groups such that transaction processing is not arrested). Regards, -Krish Krish Hariharan President/Executive Architect, Quasar Database Technologies, LLC http://www.linkedin.com/in/quasardb _____ From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Prasad Sent: Wednesday, January 16, 2008 5:01 PM To: Mir M. Mirhashimali Cc: oracle-l Subject: Re: df shows 100% on a empty filesystem- Hi Mir, This db is on solaris 8 . it do not have a lost and found . The du -a shows me the correct picture but the problem is i can not take down the db or unmount the file system now. thanks -Prasad On Jan 16, 2008 3:51 PM, Mir M. Mirhashimali <mhyder_at_rice.edu> wrote: Prasad, I was in a similar situation recently. even after moving files from the directory it was showing 100%. check "lost and found" or ".snapshot" i am on redhat linux so not sure if this applies to you. Hope this Helps Mir Prasad wrote:Received on Wed Jan 16 2008 - 23:06:47 CST
> The database is 10gR2 db running on solaris 8 .. There was archiver
> hung on archlogs01 filesystem . I moved all the archive logs to the
> backup filesystem . however df still shows archlogs01 100% filled.
> du shows correctly
> du -a /ofs/archlogs01
> 4 /ofs/archlogs01/oracle/PLLS
> 6 /ofs/archlogs01/oracle
> 10 /ofs/archlogs01
> gdf -h shows incorrect
> gdf -h | grep /ofs/archlogs01
> /dev/dsk/c2t0d11s0 2.9G 2.9G 2.0k 100% /ofs/archlogs01
> fuser -c do not show me any active process running on /ofs/archlogs01
> archive log stop also didnt help .
> Any suggestions!