Home » Infrastructure » Linux » Problem with File Retention Script and Cron Job
Problem with File Retention Script and Cron Job [message #199650] Wed, 25 October 2006 10:35 Go to next message
kumarsz
Messages: 180
Registered: March 2005
Location: canada
Senior Member

Hi,
I want to delete files which are more than 4 (its' variable) days older
i have created following script, however i am facing 3 problems with that

Problem 1
before i add my job in the crontab there were other scripts in the crontab as well
i added my job using
crontab -e

however i found that my job is not at all executing
(Note : all other jobs were added with oracle as login and i too logged in as oracle only)

what could be the reason?

Problem 2
if i execute the script manually and it deletes many files, following entry is made in logfile as well as the mail i received
/app/oracle/rep904/reports/cache - 0 txt files removed
/app/oracle/rep904/reports/cache - 0 pdf files removed

Problem 3

Also don't know which files are being removed, please see the results below

export RETAIN_TIME=4

[oracle@pulprds001 cache]$ ls -ltr|head -30
total 167712
-rw-r--r-- 1 oracle oinstall 408 Oct 19 16:29 36139676.txt
-rw-r--r-- 1 oracle oinstall 7343191 Oct 19 18:59 47099083.txt
-rw-r--r-- 1 oracle oinstall 35146 Oct 19 19:03 62101240.pdf
-rw-r--r-- 1 oracle oinstall 3269 Oct 19 19:08 60474855.pdf
-rw-r--r-- 1 oracle oinstall 2964 Oct 19 19:13 09929277.pdf
-rw-r--r-- 1 oracle oinstall 1068883 Oct 19 19:27 14809403.pdf
-rw-r--r-- 1 oracle oinstall 281617 Oct 19 19:34 35734495.txt
-rw-r--r-- 1 oracle oinstall 19681 Oct 19 20:09 38442774.pdf
-rw-r--r-- 1 oracle oinstall 66 Oct 19 20:09 25581605.txt
-rw-r--r-- 1 oracle oinstall 66 Oct 19 20:09 42807848.txt
-rw-r--r-- 1 oracle oinstall 288356 Oct 19 20:09 09343289.pdf
-rw-r--r-- 1 oracle oinstall 90331 Oct 19 20:09 53462675.txt
-rw-r--r-- 1 oracle oinstall 14313 Oct 19 20:09 08095837.txt
-rw-r--r-- 1 oracle oinstall 88614 Oct 19 20:09 23510881.txt
-rw-r--r-- 1 oracle oinstall 92282 Oct 19 20:09 27137068.txt
-rw-r--r-- 1 oracle oinstall 7438 Oct 19 21:09 00745314.pdf
-rw-r--r-- 1 oracle oinstall 35417 Oct 20 00:04 66899396.pdf
-rw-r--r-- 1 oracle oinstall 14591 Oct 20 00:07 55168650.pdf
-rw-r--r-- 1 oracle oinstall 27598 Oct 20 00:09 57517398.txt
-rw-r--r-- 1 oracle oinstall 3963 Oct 20 00:17 26300942.pdf
-rw-r--r-- 1 oracle oinstall 27145 Oct 20 00:22 24686623.pdf
-rw-r--r-- 1 oracle oinstall 3632 Oct 20 00:26 04238571.pdf
-rw-r--r-- 1 oracle oinstall 2742 Oct 20 00:31 64920027.pdf
-rw-r--r-- 1 oracle oinstall 11019 Oct 20 00:32 34208918.pdf
-rw-r--r-- 1 oracle oinstall 27144 Oct 20 00:36 66200817.pdf
-rw-r--r-- 1 oracle oinstall 3168 Oct 20 00:41 62880104.pdf
-rw-r--r-- 1 oracle oinstall 7341 Oct 20 00:46 71520925.pdf
-rw-r--r-- 1 oracle oinstall 2753 Oct 20 00:55 15036536.pdf
-rw-r--r-- 1 oracle oinstall 19737 Oct 20 01:49 24991968.pdf
[oracle@pulprds001 cache]$
[oracle@pulprds001 cache]$ /home/oracle/dbawork/admin/bin/clear_cache.sh
/app/oracle/rep904/reports/cache - 31 txt files removed

/app/oracle/rep904/reports/cache - 90 pdf files removed

[oracle@pulprds001 cache]$ /home/oracle/dead.letter... Saved message in /home/or
acle/dead.letter
[oracle@pulprds001 cache]$ ls -ltr|head -30
total 142272
-rw-r--r-- 1 oracle oinstall 9472853 Oct 20 13:40 10185414.txt
-rw-r--r-- 1 oracle oinstall 2884972 Oct 20 13:52 65655397.txt
-rw-r--r-- 1 oracle oinstall 19626 Oct 20 15:03 25879003.pdf
-rw-r--r-- 1 oracle oinstall 10019 Oct 20 15:17 22124829.pdf
-rw-r--r-- 1 oracle oinstall 2802 Oct 20 15:19 52168597.pdf
-rw-r--r-- 1 oracle oinstall 54457 Oct 20 15:21 48921262.txt
-rw-r--r-- 1 oracle oinstall 2807 Oct 20 15:21 15812312.pdf
-rw-r--r-- 1 oracle oinstall 39289 Oct 20 15:49 27732139.txt
-rw-r--r-- 1 oracle oinstall 827328 Oct 20 15:53 27762130.txt
-rw-r--r-- 1 oracle oinstall 2697 Oct 20 16:42 46357396.pdf
-rw-r--r-- 1 oracle oinstall 2697 Oct 20 16:43 16943143.pdf
-rw-r--r-- 1 oracle oinstall 2697 Oct 20 16:43 60013520.pdf
-rw-r--r-- 1 oracle oinstall 2697 Oct 20 16:44 03481448.pdf
-rw-r--r-- 1 oracle oinstall 2697 Oct 20 16:45 55729191.pdf
-rw-r--r-- 1 oracle oinstall 66 Oct 20 16:59 70528743.txt
-rw-r--r-- 1 oracle oinstall 19631 Oct 20 17:07 18507473.pdf
-rw-r--r-- 1 oracle oinstall 35421 Oct 20 17:36 63961451.pdf
-rw-r--r-- 1 oracle oinstall 7437 Oct 20 21:08 18507776.pdf
-rw-r--r-- 1 oracle oinstall 14591 Oct 20 21:15 33789594.pdf
-rw-r--r-- 1 oracle oinstall 24248 Oct 20 22:13 32096376.pdf
-rw-r--r-- 1 oracle oinstall 11116 Oct 20 22:14 01019916.pdf
-rw-r--r-- 1 oracle oinstall 2737 Oct 20 22:18 19362926.pdf
-rw-r--r-- 1 oracle oinstall 27387 Oct 20 22:19 58745904.pdf
-rw-r--r-- 1 oracle oinstall 11110 Oct 20 22:19 59677292.pdf
-rw-r--r-- 1 oracle oinstall 3989 Oct 20 22:22 36023559.pdf
-rw-r--r-- 1 oracle oinstall 27388 Oct 20 22:23 46915162.pdf
-rw-r--r-- 1 oracle oinstall 3382 Oct 20 22:24 43840504.pdf
-rw-r--r-- 1 oracle oinstall 24887 Oct 20 22:24 18900095.pdf
-rw-r--r-- 1 oracle oinstall 25047 Oct 20 22:26 16011644.pdf

now if i make a change here
export RETAIN_TIME=3

It shall remove files which are more than 3 days older, right?
However it does not remove any of the files (whereas there are files dated 20 oct 06 and i am trying all this on 25 oct 06)

Even if i try
export RETAIN_TIME=2
It is not working

Please suggest

Requirement is to remove files "Older than RETAIN_TIME"

Please refer script below
************************************ Script Start *************
#!/bin/bash
export SERVER=`uname -n | awk -F. '{ print $1 }'`
export CACHE_PATH=/app/oracle/rep904/reports/cache
export RETAIN_TIME=6
export LOGFILE=/tmp/clear_cache.log

FILE_COUNT=`find $CACHE_PATH -type f -name "*.txt" -mtime +$RETAIN_TIME|wc -l`
find $CACHE_PATH -type f -name "*.txt" -mtime +$RETAIN_TIME -exec rm {} \;
if [ $? -ne 0 ]; then
echo -e "Error removing txt files - $CACHE_PATH\n"
else
echo -e "$CACHE_PATH - $FILE_COUNT txt files removed\n"
fi
FILE_COUNT=`find $CACHE_PATH -type f -name "*.pdf" -mtime +$RETAIN_TIME|wc -l`
find $CACHE_PATH -type f -name "*.pdf" -mtime +$RETAIN_TIME -exec rm {} \;
if [ $? -ne 0 ]; then
echo -e "Error removing pds files - $CACHE_PATH\n"
else
echo -e "$CACHE_PATH - $FILE_COUNT pdf files removed\n"
fi
mail -s "Cache Clear $SERVER" dba_activity@ABC.com < $LOGFILE
************************************ Script End *************
OS is Linux RH9

Thanks in Advance
Pratap



Re: Problem with File Retention Script and Cron Job [message #204205 is a reply to message #199650] Sun, 19 November 2006 00:16 Go to previous message
prabhakarkamath
Messages: 13
Registered: February 2006
Junior Member
Try the following fixes...

Problem 1
however i found that my job is not at all executing

Fix:
Check the crontab entry whether it is properly set and getting executed at the scheduled time. Better, give crontab entry here to verify.

Problem 2
if i execute the script manually and it deletes many files, following entry is made in logfile as well as the mail i received

Fix:
I can see in the code within script which exports RETAIN_TIME
export RETAIN_TIME=6
Whenever you are running the script, are you editing the script to set the new value to RETAIN_TIME? or just exporting the new value RETAIN_TIME in the shell prompt and running?

If you are setting new value in the shell prompt, then, it will not be taken since within script there is export of RETAIN_TIME and sets new value always to 6.

So,to get the result, please check if there are files older than 6 days in the directory.

Problem 3
Also don't know which files are being removed, please see the results below

Fix:
Add the code where the list of files getting deleted will be re-directed before deletion.


find $CACHE_PATH -type f -name "*.pdf" -mtime +$RETAIN_TIME >> $LOGFILE

Hope this helps!!

Cheers
Previous Topic: setup on suse
Next Topic: sqlldr
Goto Forum:
  


Current Time: Sun Dec 04 00:10:51 CST 2016

Total time taken to generate the page: 0.07485 seconds