Re: On-line Backup

From: RungOnMt <rungonmt_at_aol.com>
Date: 1995/05/09
Message-ID: <3opc28$eh7_at_newsbf02.news.aol.com>

You asked for it, you got it...
This has been a pretty popular script, so I'll just post it in the group as well..

These two scripts are in production at our site now for a long time. They work great, and do online backups of over 20 G of data. The archives are VERY CPU intensive, because you are copying so much data, but our goal was to keep it up 24X7, not make it scream 24X7. So the users can run their queries and reports that take all night, and I don't have to knock them out to do my backups.

Any questions, give me a call (302) 457-4138

#!/bin/sh
# SCCS = _at_(#)ora_back.sh 1.30 02/15/95

# Authors: Curtis Preston

# No warranties - implied or otherwise!!!
# Use at your own risk.....etc.
#
# This script is intended to be put in an nfs mounted directory (/admin1)
# It is called by run_ora_back.sh ( later) The purpose of the
ora_back_stat
# file is so that a third party backup utility (such as SM-arch that I am
using)
# can run a script that checks that the backup completed successfully
before
# it backs up the online backup to tape
# This script has worked wonderfully, with the modifications and bug
fixes
# listed below, most of which were driven by a user group that kept
changing
# their mind as to how the wanted me to do this (skip develop db's ,
etc.)
# Chop and use away!!!!

# Date Changes (changed by)
# -----------------------------------
# 9/1/94 - Initial (Curtis Preston & Sam Binder (Coretech
Consulting))
# 09/12/94 Added some logging and took some out
# 09/13/94 Added a section to sleep if not enough file sys space
# 09/14/94 Changed host copy command to compress -c
# Also deleted all sections to do w/ old compress (sleep,
etc)
# 09/14/94 Redirected stderr of sqldba commands to /tmp/oraclebu, and
# also sent them to stdout if they appear
# 09/16/94 Changed "Connect /" to "connect internal", because program
# is now going to be run by oracle to solve multiple group
# permissioning problems
# Also redirected stderr of sqldba commands into pipe to
# grep -v the useless error from compress "toolong:
permission..."
# 10/3/94 Added section to specify production of non-production
backup
# 10/4/94 Added more echoing, reporting, etc.
# Also fixed bug in controlfile section
# 10/13/94 Added section that will check the third field in the /etc
# oratab file and set DB_UP to 1 and LOGS_UP to 0,
# which is the only condition under which the DB will not
# be backed up - Allows development databases to be excluded
# Also fixed DB_UP and LOGS_UP variables by using grep -v
grep
# 10/18/94 Changed section of script that did a cp of files if
database
# was down to make it do a compress -c like the rest of the
script
# 01/03/95 Previous fix to check for archiving up didnt work so well
# totally redid that whole section. Script became
complicated
# enough that I had to make it a case statement also (wcp)
# Also changed section of script that prints out error
messages
# If things don't go right...
# 01/17/95 New case statement had error in logic. Changed so that
will
# Skip backup of dbase, if there is an N in the
/etc/oratab(wcp)
# 01/18/95 Found missing slash on log file line and also added if
statement
# that only checks for down database status if no "N" in
oratab
# 02/15/95 Added new case that would not backup a database if
#
# This script will backup all tablespaces, redo files, and control files.
# The databases backed up come from the /etc/oratab file.
#
# Backs up Hot databases and Down databases as a precaution
# Setup Oracle variables

HOST=`hostname`

EXIT0()
{
echo "COMPLETE" >/appl/oracle/$HOST.ora_back_stat }

EXIT1()
{
echo "NOT COMPLETE" >/appl/oracle/$HOST.ora_back_stat }

error_but_no_exit()
{
echo "ERROR" >/appl/oracle/$HOST.ora_back_temp_stat }

log_errors()
{
if [ $? -gt 0 ]
then

    echo "  **********************ERROR!***********************"\
        |tee -a /tmp/oraclebu
    cat /tmp/tablespace.out|sed 's/^/  /'\
        |tee -a /tmp/oraclebu
    echo "  ***************************************************"\
        |tee -a /tmp/oraclebu

error_but_no_exit
fi
}

#"zero out the log"

echo "OKSOFAR" >/appl/oracle/$HOST.ora_back_temp_stat

if [ $# -gt 1 ]
then

        echo "Usage: $0 [prod/devel]"
        exit 1
        EXIT1

DBTEMP=/tmp
BACKUPDIR=/backups

    HOME=/appl/oracle
    PATH=$PATH:$ORACLE_HOME/bin:/ORACLE_BASE/local/bin:/admin1/bin
    PATH=$PATH:/appl/oracle/bin:/appl/oracle/local/bin
    PATH=$PATH:/appl/oracle/product/6.0.37/bin:/bin:/bin/posix
    PATH=$PATH:/prod1/corpdba/bin:/usr/bin:/usr/contrib/bin
    PATH=$PATH:/usr/informix/bin:/usr/local/bin:usr/informix
    PATH=$PATH:/appl/oracle:/appl/oracle/local/bin
    PATH=$PATH:/appl/oracle/product/7.0.13/bin
    PATH=$PATH:/appl/oracle/bin:/bin/posix:/bin:/usr/bin:/usr/contrib/bin

    ORATERMPATH=:/appl/oracle/product/6.0.37/forms30/admin/resource     ORAKITPATH=/appl/oracle
    ORACLE_BASE=/appl/oracle
    ORACLE_PATH=$ORACLE_BASE/local/naps/sql7:$ORACLE_BASE/aps6/sql     MENU5PATH=/appl/oracle:$ORACLE_HOME/forms30/admin/resource

export DBTMP PATH HOME ORAKITPATH MENU5PATH ORATERMPATH ORACLE_BASE export ORACLE_PATH

#First, maintain the size of the /tmp/oraclebu file

tail -3000 /tmp/oraclebu >/tmp/oraclebu2 mv /tmp/oraclebu2 /tmp/oraclebu

# First we must determine the existance of the /etc/oratab file

if test -s /etc/oratab
then

    echo "------------------------------------"\
        | tee -a /tmp/oraclebu
    echo "\nStarting Oracle backup: `date '+%X %D'`" \
        | tee -a /tmp/oraclebu

else

    echo "------------------------------------"\
        | tee -a /tmp/oraclebu

echo "COULD NOT FIND /etc/oratab - ABORTING ORACLE BACKUP: `date '+%X %D'`"\

        | tee -a /tmp/oraclebu
    exit 1
    EXIT1
fi

if [ $# = 1 ]
then
  if [ "$1" = "prod" ]
  then
    #get a list of production databases, which all begin with "p"     SID_LIST=`cat /etc/oratab | grep -v '^\#'|grep "^p"`   else
    #get a list of development databases, which all begin with other than "p"

SID_LIST=`cat /etc/oratab | grep -v '^\#'|grep -v "^p"` fi
else #There are no arguments

#get a list of all databases
SID_LIST=`cat /etc/oratab | grep -v '^\#'` #list all sids fi

#TOOLS:/u00/ORACLE/prod/tools01prod.dbf
#
#######################################################
#Here's a section I thought up if size ever becomes a problem
#######################################################
#LINE=`head -1 /backups/$ORACLE_SID/tablespace_list|tail -1`
#
# OFILE=`echo $LINE|cut -d: -f2`
# FILE=`echo $LINE|cut -d: -f2|sed 's-.*\/--g'`
# CFILE=/backups/$ORACLE_SID/$FILE
# SIZE_OF_OFILE=`du -s $OFILE| awk ' {print $1}'`
# SIZE_OF_CFILE=`du -s ${CFILE}.Z|sort -rn\
# |head -1|awk ' {print $1}'`
# if [ $SIZE_OF_CFILE = 0 ]
# then
# SIZE_OF_CFILE=$SIZE_OF_OFILE
# fi
# COMPRESSION=`expr $SIZE_OF_OFILE / $SIZE_OF_CFILE`
#
#BLOCKS_AVAIL=`bdf /backups|tail -1|awk '{print $4}'`
#FREE=`expr $BLOCKS_AVAIL + $SIZE_OF_CFILE`
#NEED=`expr $SIZE_OF_OFILE / $COMPRESSION`
#if [ $NEED -gt $FREE ]
#then
# echo "HEY NO ROOM!"
#fi
#######################################################

for LINE in $SID_LIST #for each ORACLE_SID in oratab do

ORACLE_SID=`echo $LINE | awk -F: '{ print $1 }'` export ORACLE_SID
VERSION=`echo $LINE | awk -F: '{ print $2 }' | sed 's/.*\///g'|cut -d'.' -f1`

if [ "$VERSION" -eq 6 ]
then

ORACLE_HOME=/appl/oracle/product/6.0.37 APPL_TOP=""
export ORACLE_HOME APPL_TOP

#This next section will create a "1" if ARCHIVELOG is set to NO, which #would mean that the online backup can not be completed sqldba <<EOF >/tmp/archivemode.out
set echo on
connect internal;
archive log list;
exit;
EOF LOGMODE=`grep 'Database log mode.*ARCHIVELOG' /tmp/archivemode.out\

| grep -v NO | wc -l |sed 's/ //g'`

else #This is NOT a version 6 database

ORACLE_HOME=/appl/oracle/product/7.0.13 APPL_TOP=/appl/finmgr
export ORACLE_HOME APPL_TOP

#This next section will create a "1" if ARCHIVELOG is set to NO, which #would mean that the online backup can not be completed sqldba <<EOF >/tmp/archivemode.out
set echo on
connect internal;
select log_mode from v\$database;
exit;
EOF LOGMODE=`grep 'ARCHIVELOG' /tmp/archivemode.out|grep -v NO\

| wc -l |sed 's/ //g'`

  if [ $LOGMODE -eq 1 ]
  then #Archiving is on
    LOGS_UP=20
  else #Archiving is off
    LOGS_UP=10
  fi

DB_PROCESSES=`ps -ef | grep 'ora_...._...' | grep $ORACLE_SID|grep -v grep
\

|wc -l | sed 's/ //g'` > /dev/null

  if [ $DB_PROCESSES -ge 4 ]
  then #Database is UP
    DB_UP=2
  else #Database is DOWN
    DB_UP=1
  fi

#Section below added to allow developers to put an 'N' in the /etc/oratab
file
#Which will exclude that database from backups

STARTUP_AUTO=`grep "^$ORACLE_SID" /etc/oratab |cut -d: -f3| sed 's/ //g'`

  if [ "$STARTUP_AUTO" = "Y" ]
  then
    START=200
  else
    START=100
  fi

DB_STAT=`expr $LOGS_UP + $DB_UP + $START`

case $DB_STAT in

222) #If DB is up and LOGS are on, and a "Y" is in oratab

    echo "  -----------------------------"| tee -a /tmp/oraclebu

echo " Backup of $ORACLE_SID Started"| tee -a /tmp/oraclebu

mkdir $BACKUPDIR/$ORACLE_SID 1>/dev/null 2>/dev/null sqldba << EOF > /tmp/tablespace.out 2>>/tmp/oraclebu set echo on
connect internal;
spool /tmp/$ORACLE_SID.tablespace_list
select tablespace_name, file_name from sys.dba_data_files order by tablespace_name;
exit
EOF log_errors

sqldba << EOT > $BACKUPDIR/$ORACLE_SID/logs 2>>/tmp/oraclebu set echo on
connect internal;
select * from v\$logfile;
exit
EOT log_errors

cat /tmp/$ORACLE_SID.tablespace_list |grep -e '\.dbf' -e '\.idx' |sed 's/
*
/ /g'\

| sed 's/ /:/g'| sed 's/://2' > $BACKUPDIR/$ORACLE_SID/tablespace_list

echo "Tablespaces backed up by this script can be found in: \c" \

|tee -a /tmp/oraclebu
echo "$BACKUPDIR/$ORACLE_SID/tablespace_list" |tee -a /tmp/oraclebu

# Backup the tablespaces
cat $BACKUPDIR/$ORACLE_SID/tablespace_list|while read TBSLINE do

      FILE=`echo $TBSLINE | awk -F: '{print $2}'`
      TBS=`echo $TBSLINE | awk -F: '{print $1}'`
      FILENAME=`echo $FILE | sed 's/.*\///g'`

      echo "Backing up $FILE." | tee -a /tmp/oraclebu

      #Section below changed from "host cp FILE" to "host compress -c
FILE"
      #To speed up backup and conserve space in BACKUPDIR

      sqldba << EOF > /tmp/tablebu.$ORACLE_SID 2>>/tmp/oraclebu

set echo on
connect internal;

alter tablespace $TBS begin backup;
host compress -c $FILE > $BACKUPDIR/$ORACLE_SID/$FILENAME.Z
alter tablespace $TBS end backup;

exit
EOF log_errors

cat /tmp/tablebu.$ORACLE_SID|sed 's/.*SQLDBA>//g'|grep -v "SQL\*DBA: [RV]"\

        |grep -v "Copyright (c) Oracle"\
        |grep -v "ORACLE.* Server Release"\
        |grep -v "ORACLE RDBMS V"\
        |grep -v "With the procedural option"\
        |grep -v "transaction processing option"\
        |grep -v "PL/SQL [RV]"\
        |sed 's/  *//'\
        |grep -v 'Connected\.$'\
        |grep -v 'Statement processed\.$'\
        |grep -v 'SQL\*DBA complete\.'\
        |sed '/^$/d' >/tmp/out.file

        if [ `cat /tmp/out.file|wc -l|sed 's/ //g'` -gt 0 ]
        then
           echo  "  *********************ERROR!***********************"\
                        |tee -a /tmp/oraclebu
           cat /tmp/out.file|sed 's/^/  /' \
                        |tee -a /tmp/oraclebu
           echo "  ***************************************************"\
                        |tee -a /tmp/oraclebu
           error_but_no_exit
        fi


      cd /$BACKUPDIR/$ORACLE_SID

      done #done with all tablespaces

# Create a list of all the controlfiles for this database

if [ "$VERSION" -eq 6 ]
then

      grep control /$ORACLE_HOME/dbs/init$ORACLE_SID.ora \
                |sed 's/        //g'|sed 's/ //g'|sed 's/,//g' |sed
's/)//g'\
                |sed 's/control_files=(//g' \
                 >$BACKUPDIR/$ORACLE_SID/controlfile_list
    else
      grep control /appl/oracle/admin/$ORACLE_SID/pfile/config.ora\
                 | sed 's/      //g'|sed 's/ //g'|sed 's/,//g' |sed
's/)//g'\
                 |sed 's/control_files=(//g' \
                  >$BACKUPDIR/$ORACLE_SID/controlfile_list

## Time to do the checkpoint, switch the logfile and backup control

echo "Switching logfile and backing up controlfiles with Oracle"\

|tee -a /tmp/oraclebu

sqldba << EOF >/tmp/controlfilebu 2>>/tmp/oraclebu set echo on;
connect internal;
alter system checkpoint;
alter system switch logfile;
alter database backup controlfile to '$BACKUPDIR/$ORACLE_SID/controlfile'; exit;
EOF log_errors

compress -f $BACKUPDIR/$ORACLE_SID/controlfile

log_errors

cat /tmp/controlfilebu|sed 's/.*SQLDBA>//g'|grep -v "SQL\*DBA: [RV]"\

        |grep -v "Copyright (c) Oracle"\
        |grep -v "ORACLE.* Server Release"\
        |grep -v "ORACLE RDBMS V"\
        |grep -v "With the procedural option"\
        |grep -v "transaction processing option"\
        |grep -v "PL/SQL [RV]"\
        |sed 's/  *//'\
        |grep -v 'Connected\.$'\
        |grep -v 'Statement processed\.$'\
        |grep -v 'SQL\*DBA complete\.'\
        |sed '/^$/d' >/tmp/out.file

        if [ `cat /tmp/out.file|wc -l|sed 's/ //g'` -gt 0 ]
        then
           echo  "  *********************ERROR!***********************"\
                        |tee -a /tmp/oraclebu
           cat /tmp/out.file|sed 's/^/  /'\
                        |tee -a /tmp/oraclebu
           echo "  ***************************************************"\
                        |tee -a /tmp/oraclebu
           error_but_no_exit

        fi

echo " Backup of $ORACLE_SID Completed" | tee -a /tmp/oraclebu ;;

211|111) #Database is completely down (even if N in oratab)

if [ $DB_STAT = 211 ]
then

TOT_DB_PROCESSES=`ps -ef | grep 'ora_...._...' |grep -v grep | wc -l
|\

sed 's/ //g'` > /dev/null

      echo "  -----------------------------"| tee -a /tmp/oraclebu


      if [ $TOT_DB_PROCESSES -gt 0 ]
      then
         error_but_no_exit
      fi

    fi
    backup_offline()
    {
      echo " Backing up $ORACLE_SID offline using last setupfile created"\

| tee -a /tmp/oraclebu

      mkdir $BACKUPDIR/$ORACLE_SID 1>/dev/null 2>/dev/null
      # Backup the tablespaces
      cat $BACKUPDIR/$ORACLE_SID/tablespace_list | while read TBLINE
      do
        FILE=`echo $TBLINE | awk -F: '{print $2}'`
        FILENAME=`echo $FILE | sed 's/.*\///g'`
        compress -c $FILE >$BACKUPDIR/$ORACLE_SID/$FILENAME.Z
      done   #done with the table space

      cat $BACKUPDIR/$ORACLE_SID/controlfile_list |while read CTRLLINE
      do
        FILE=`echo $CTRLLINE | awk '{print $1}'`
        FILENAME=`echo $FILE | sed 's/.*\///g'`
        compress -c $FILE> $BACKUPDIR/$ORACLE_SID/controlfile.Z
        log_errors
      done #done with all the control files
      echo "  Backup of $ORACLE_SID Completed" | tee -a /tmp/oraclebu

}

HOST=`hostname`
BACKUP_TYPE=`cat /appl/oracle/ora_backup_type|sed 's/ //g'`

    if [ "$STARTUP_AUTO" = "Y" ]
    then
      backup_offline
    else

      if [ "$BACKUP_TYPE" = "HOT" ]
      then
        echo "  Backup of $ORACLE_SID SKIPPED!" | tee -a /tmp/oraclebu
        echo "  (ORACLE_SID $ORACLE_SID was down during a hot backup,"\
           | tee -a /tmp/oraclebu
        echo "   and has an \"N\" in the /etc/oratab)"\
           | tee -a /tmp/oraclebu
      else
        backup_offline
      fi
    fi
     ;;

   212)
      echo "  -----------------------------"| tee -a /tmp/oraclebu
      echo "  Backup of $ORACLE_SID FAILED!" | tee -a /tmp/oraclebu
      OK=NO
      echo "    ***** $ORACLE_SID WAS UP, BUT ARCHIVING WAS OFF! *****" \
           | tee -a /tmp/oraclebu
      date >/tmp/ora_errors
           error_but_no_exit

;;

112|122)

      echo "  -----------------------------"| tee -a /tmp/oraclebu
      echo "  Backup of $ORACLE_SID SKIPPED!" | tee -a /tmp/oraclebu
      OK=NO
      echo "   (ORACLE_SID $ORACLE_SID has an \"N\" in the /etc/oratab)"\
           | tee -a /tmp/oraclebu

;;

  #The case 121, the only one not addressed as a case statement,
  #should never happen since the logs can not be up if the
  #database is down, so if this happens, there is an error in the logic

  *) echo "THERE IS AN ERROR IN THE LOGIC OF $0 !!!!"|tee -a /tmp/oraclebu
     echo "(Case statement read $DB_STAT)"|tee -a /tmp/oraclebu

;;
esac

done #done with all $ORACLE_SIDS

echo "Backup of Oracle is now completed" |tee -a /tmp/oraclebu date | tee -a /tmp/oraclebu

if [ `cat /appl/oracle/$HOST.ora_back_temp_stat` != "OKSOFAR" ] then
EXIT1
else
EXIT0
fi

#!/bin/sh
#
# This is the script to run the above script.....
# Same risk factor applies!!!!

# SCCS = _at_(#)run_ora_back.sh 1.15 02/15/95
#
# Authors: Curtis Preston
# Date Changes (changed by)
# -----------------------------------
# 9/1/94 - Initial (Curtis Preston & Sam Binder (Coretech
Consulting))
# 09/8/94 - Added weekly shutdown of oracle database for full backup
# 09/12/94 - Added some more logging
# 09/14/94 - Added killing of leftover oracle processes, and starting
# of offline ora_back.sh for shutdown days
# 09/16/94 Changed it to have oracle run backup, not backup id
# because of multiple file group permissioning problems
# 10/3/94 Added section to go with new "prod" option for ora_back.sh
# Script will run production backup only if $0 is
# run_ora_back.prod.sh, devel only if $0 ..devel, all if
# 10/4/94 Increased logging by changing logfile to include $ARGUMENT
# 1/8/94 Added loop to restart both 6 & 7 databases (wcp)
# 1/24/94 Removed option that was added 10/3 & 10/4
# 2/13/94 Added PATH variable and changed variable PATH to PPATH
#
####################################################################

#This script is to run /admin1/bin/ora_back.sh as oracle, since
#SM-arch cannot pass arguments to a script yet

PATH=$PATH:$ORACLE_HOME/bin:/ORACLE_BASE/local/bin:/admin1/bin
PATH=$PATH:/appl/oracle/bin:/appl/oracle/local/bin
PATH=$PATH:/appl/oracle/product/6.0.37/bin:/bin:/bin/posix
PATH=$PATH:/prod1/corpdba/bin:/usr/bin:/usr/contrib/bin
PATH=$PATH:/usr/informix/bin:/usr/local/bin:usr/informix
PATH=$PATH:/appl/oracle:/appl/oracle/local/bin
PATH=$PATH:/appl/oracle/product/7.0.13/bin
PATH=$PATH:/appl/oracle/bin:/bin/posix:/bin:/usr/bin:/usr/contrib/bin

ORATERMPATH=:/appl/oracle/product/6.0.37/forms30/admin/resource ORAKITPATH=/appl/oracle
ORACLE_PATH=$ORACLE_BASE/local/naps/sql7:$ORACLE_BASE/aps6/sql MENU5PATH=/appl/oracle:$ORACLE_HOME/forms30/admin/resource export DBTMP PATH HOME ORAKITPATH MENU5PATH ORATERMPATH ORACLE_BASE export ORACLE_PATH

FULL_BACKUP_DAY=Sun
HOST=`hostname`

DAY=`date +%a`

if [ $DAY = $FULL_BACKUP_DAY ]
then

echo "COLD" >/appl/oracle/ora_backup_type

if test -f /etc/shutdown.d/cm_stop.prompt #Will test if concurrent manager

  then                                      #script is there, if so, it

will

echo "Shutting down concurrent manager." >>/tmp/ora_back.out 2>&1 /etc/shutdown.d/cm_stop.prompt #run it fi

echo "Shutting down database." >>/tmp/ora_back.out 2>&1 /etc/shutdown.d/dbshut.immed

sleep 60

  for PID in `ps -ef|grep 'ora_...._...'|awk '{print $2}'`   do
    echo "Killing leftover database process" >>/tmp/ora_back.out 2>&1     ps -ef|grep -v grep |grep " $PID "
    kill $PID
  done

echo "SHUTDOWN DATABASE FOR FULL BACKUP on `date`" \

>>/tmp/ora_back.out 2>&1

echo "/bin/su oracle -c /admin1/bin/ora_back.sh" \

>>/tmp/ora_back.out 2>&1

/bin/su oracle -c "/admin1/bin/ora_back.sh " \

>>/tmp/ora_back.out 2>&1

#Start the database(s) back up

for PPATH in `cat /etc/oratab|grep -v '^#'|cut -d: -f2|sed 's-.*\/--g'\

        |sort -u`
  do
    /bin/su oracle -c /appl/oracle/product/$PPATH/bin/dbstart >/tmp/dbstart.out
  done

if test -f /etc/rc.conc_man #Will test if concurrent manager

  then                                      #script is there, if so, it
will
    /etc/rc.conc_man        #run it

else

echo "HOT" >/appl/oracle/ora_backup_type

echo "/bin/su oracle -c /admin1/bin/ora_back.sh " \

>>/tmp/ora_back.out 2>&1

/bin/su oracle -c "/admin1/bin/ora_back.sh " \

>>/tmp/ora_back.out 2>&1

Remember, My views [QUITE OFTEN] do not reflect those of my company... :)

=====================Your mileage may vary=========================
       Although I do consider myself a UNIX professional, always keep in

mind that

a professional does not mean you're good at it, It just means you get paid for it!!!

P.S. In case you're wondering, the login is significant...... As any good Sysadmin, I
am rungonmt (that is, running on empty....) LOL Received on Tue May 09 1995 - 00:00:00 CEST

This message: [ Message body ]
Next message: Dag Eggemoen: "Alternate reporting tool for Oracle ?"
Previous message: Sergio Bertolini: "SQL*Net and Redundant Networks"
In reply to: Chrisi: "On-line Backup"
Next in thread: Scott Urman: "Re: database link problem"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

Original text of this message