Oracle FAQ Your Portal to the Oracle Knowledge Grid
HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US
 

Home -> Community -> Mailing Lists -> Oracle-L -> RMAN fails because of corrupt block in seemingly free space in SYSTEM

RMAN fails because of corrupt block in seemingly free space in SYSTEM

From: <Uldis.Pavuls_at_tietoenator.com>
Date: Tue, 2 Jan 2007 12:58:37 +0200
Message-ID: <1FABB43D1E997C49B412A3008386ADB9024E1ABD@dino.eu.tieto.com>


As it seems, the corruption should be in the free space in the SYSTEM tablespace. I just have no clue what to do with that. (Besides, wouldn't RMAN just skip the really free space?.) If it was for ordinary tablespace I'd worry less, altogether this corruption do not permit to make a valid backup - with all the consequences.
(Notice, they are raw devices.)

Just facts in the begining:
System: Oracle 9.2.0.7.0-64bit, EE

System name:	AIX
Release:	3
Version:	5

RMAN's spool:
RMAN>
connected to target database: CMS (DBID=216621080) using target database controlfile instead of recovery catalog RMAN> 2> 3> 4> 5> 6>
allocated channel: t1
channel t1: sid=30 devtype=DISK
Starting backup at 28-DEC-06
channel t1: starting full datafile backupset channel t1: specifying datafile(s) in backupset input datafile fno=00001 name=/dev/rcmssystem .......
input datafile fno=00004 name=/dev/rcmsusers channel t1: starting piece 1 at 28-DEC-06 channel t1: finished piece 1 at 28-DEC-06 piece handle=/mnt/backup/tandd_srv/CMS/CMS_610371224_12_1 comment=NONE channel t1: starting piece 2 at 28-DEC-06 channel t1: finished piece 2 at 28-DEC-06 piece handle=/mnt/backup/tandd_srv/CMS/CMS_610371224_12_2 comment=NONE channel t1: starting piece 3 at 28-DEC-06 released channel: t1

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on t1 channel at 12/28/2006 11:45:00 ORA-19566: exceeded limit of 0 corrupt blocks for file /dev/rcmssystem

From alert-log (trc-file contains pretty the same): Thu Dec 28 11:33:16 2006
Errors in file /ora/admin/CMS/udump/cms_ora_1003618.trc: ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory Additional information: 4
***
Corrupt block relative dba: 0x0040ffff (file 1, block 65535) Fractured block found during backing up datafile Data in bad block -
 type: 0 format: 2 rdba: 0x0000ffff
 last change scn: 0x0000.00000000 seq: 0x1 flg: 0x05  consistency value in tail: 0x0074117a
 check value in block header: 0xfef9, computed block checksum: 0x9  spare1: 0x0, spare2: 0x0, spare3: 0x0
***
Reread of blocknum=65535, file=/dev/rcmssystem. found same corrupt data Thu Dec 28 11:45:01 2006
Errors in file /ora/admin/CMS/udump/cms_ora_1765522.trc: ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory Additional information: 4

Queries to find where the corruption is:

select T.NAME "T-space", D.NAME "Filename", C."FILE#" "FileId", C."BLOCK#" "BlockId", C.BLOCKS,
C."CORRUPTION_CHANGE#" "Corr.SCN", C.CORRUPTION_TYPE from SYS."V_$DATABASE_BLOCK_CORRUPTION" C, SYS."V_$DATAFILE" D, SYS."V_$TABLESPACE" T
where C."FILE#" = D."FILE#"
and D."TS#" = T."TS#"
order by 1, 2, 3, 4 ;
(Output in CSV)

SYSTEM;/dev/rcmssystem;1;65535;1;0;FRACTURED SYSTEM;/dev/rcmssystem;1;65536;9;0;FRACTURED

select OWNER,SEGMENT_NAME,PARTITION_NAME,SEGMENT_TYPE,BLOCKS from SYS.DBA_EXTENTS
where FILE_ID=1
and BLOCK_ID <=65535 and (BLOCK_ID+BLOCKS-1)>=65536;  no rows selected

select TABLESPACE_NAME,FILE_ID,BLOCKS
from SYS.DBA_FREE_SPACE where FILE_ID=1 and BLOCK_ID <=65535 and (BLOCK_ID+BLOCKS-1)>=65536;
(Output in CSV)

SYSTEM;1;87420 I haven't got the result of "DBMS_SPACE_ADMIN.TABLESPACE_VERIFY('SYSTEM')" by now, though (I'm not on-site there - they are our "application-use only" customers). Explanation and ideas will be dearly appreciated. TIA,
Uldis

Uldis Pāvuls
DBA, TietoEnator
Lāčplēša 41, Rīga, Latvja, LV 1011
phone +371 7 286 660, fax +371 7 771 313 mailto:U.Pavuls_at_tietoenator.com
http://www.tietoenator.lv
http://www.tietoenator.com

--
http://www.freelists.org/webpage/oracle-l
Received on Tue Jan 02 2007 - 04:58:37 CST

Original text of this message

HOME | ASK QUESTION | ADD INFO | SEARCH | E-MAIL US