Re: datapump export to pipe

From: Tony Adolph <tony.adolph.dba_at_gmail.com>
Date: Mon, 12 Oct 2009 17:07:02 +1300
Message-ID: <4a38d9060910112107n5dd4929bp951bc7040d7ab6c0_at_mail.gmail.com>

Hi All,

Thanks for the ideas.

I thought I'd got it with the following sh script, but it looks like exppd closes and re-opens a dump file from time to time,.. or fuser is lying to me :-)

Here's the script (or the interesting bits):

rm -f ${DMPFILE}* $LOGFILE
echo "`date` expdp started ..."
expdp $CREDENTIALS \

      DIRECTORY=DP_EXP_DIR \
      DUMPFILE=${DMPFILE}_%U \
      FULL=Y \
      LOGFILE=$LOGFILE \
      FILESIZE=$FILESIZE \
      JOB_NAME=${ORACLE_SID}_SPLIT_EXP 2>&1 >/dev/null &

EXP_PID=$! echo "`date` expdb running with PID $EXP_PID waiting to gzip dmp files" while [ `ps -ef|grep $EXP_PID|grep -v grep|wc -l` -gt 0 ] ; do   DMP_FILES=`ls ${DMPFILE}_* 2>/dev/null|grep -v ".gz"|grep -v ${DMPFILE}_01 `
  for F in $DMP_FILES ; do
    if [ -f ${F}.gz ] ; then

       echo "`date` $F being gzipped"
    else

      if [ `/sbin/fuser $F|wc -l` -eq 0 ] ; then
        echo "`date` gzipping $F"
        gzip $F &
      else
        echo "`date` skipping $F - still dumping"
      fi

    fi
  done
  echo "`date` sleeping..."
  sleep 60
done

# job's finished, but may be some dmp files to compress... DMP_FILES=`ls ${DMPFILE}_* 2>/dev/null|grep -v ".gz"` for F in $DMP_FILES ; do
  if [ -f ${F}.gz ] ; then
    echo "`date` $F being gzipped"
  else
    echo "`date` gzipping $F"
    gzip $F
  fi
done

if [ -f $LOGFILE ] ; then
  if [ `grep "Master table" $LOGFILE|grep successfully|wc -l` -gt 0 ] ; then     echo "`date` looks good"
  else
    echo "`date` OH BUGGER expdb failed"   fi
else
  echo "`date` expdb failed to create a logfile" fi

I set the filesize = 64M which was the max extent size and created loads of small file for testing.

But expdp failed because a file got gzipped before expdp had finished with it. I haven't included a file size check, only fuser as you can see above.

Any other ideas? I was thinking about doing a cross-check in the database to see if the file write was complete, perhaps in expdp's control table, but haven't found where yet.

Regards
Tony

--
http://www.freelists.org/webpage/oracle-l

Received on Sun Oct 11 2009 - 23:07:02 CDT