Re: July CPU on 11.2.0.2

From: Robert Hanuschke <robert.hanuschke_at_googlemail.com>
Date: Sat, 20 Aug 2011 21:02:07 +0200
Message-ID: <CAL6WDh2FQ7L_ou6sLwoOiB=BCxBKJXoX7fHuboQ89gFJnZjyyQ_at_mail.gmail.com>

Hi,

I ran into a similar problem. I'm using udev though, so the error messages don't match 1:1. The ORA-600 is there with the same argument, though. I think the source of the problem could very well be the same and might be worth checking out.

I get the error when creating a database on a newly set up RAC for a current project after I applied the PSU for GI and DB home.

Unsuccessfuly try:

[oracle_at_rac02b ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Sat Aug 20 20:34:06 2011

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 626327552 bytes

Fixed Size                  2229080 bytes
Variable Size             184552616 bytes
Database Buffers          436207616 bytes
Redo Buffers                3338240 bytes

SQL> create database DS0P
2 USER SYS IDENTIFIED BY manager1
USER SYSTEM IDENTIFIED BY manager1
EXTENT MANAGEMENT LOCAL
DEFAULT TEMPORARY TABLESPACE TEMP
UNDO TABLESPACE UNDOTBS1
DEFAULT TABLESPACE USERS
CHARACTER SET WE8ISO8859P15
NATIONAL CHARACTER SET AL16UTF16
ARCHIVELOG
FORCE LOGGING; 3 4 5 6 7 8 9 10 11
create database DS0P
*
ERROR at line 1:
ORA-03113: end-of-file on communication channel Process ID: 28206
Session ID: 1 Serial number: 3

Corresponding alert log entry:

Sat Aug 20 20:35:46 2011
create database DS0P
USER SYS IDENTIFIED BY *USER SYSTEM IDENTIFIED BY *EXTENT MANAGEMENT LOCAL DEFAULT TEMPORARY TABLESPACE TEMP
UNDO TABLESPACE UNDOTBS1
DEFAULT TABLESPACE USERS
CHARACTER SET WE8ISO8859P15
NATIONAL CHARACTER SET AL16UTF16
ARCHIVELOG
FORCE LOGGING
Sat Aug 20 20:35:46 2011
ORA-15025: could not open disk "/dev/asm-disk4" ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied Additional information: 9
ORA-15025: could not open disk "/dev/asm-disk6" ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied Additional information: 9
ORA-15025: could not open disk "/dev/asm-disk5" ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied Additional information: 9
ORA-15025: could not open disk "/dev/asm-disk7" ORA-27041: unable to open file
Linux-x86_64 Error: 13: Permission denied Additional information: 9
SUCCESS: diskgroup DATA was mounted
Errors in file /u01/app/oracle/diag/rdbms/ds0p/DS0P/trace/DS0P_ora_28310.trc (incident=1817):
ORA-00600: internal error code, arguments: [kfioTranslateIO03], [], [], [], [], [], [], [], [], [], [], []
Incident details in:
/u01/app/oracle/diag/rdbms/ds0p/DS0P/incident/incdir_1817/DS0P_ora_28310_i1817.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ERROR: unrecoverable error ORA-600 raised in ASM I/O path; terminating process 28310
Sat Aug 20 20:35:48 2011
Dumping diagnostic data in directory=[cdmp_20110820203548], requested by (instance=1, osid=28310), summary=[incident=1817]. Sat Aug 20 20:36:24 2011
Sweep [inc][1817]: completed
Sweep [inc2][1817]: completed
Sat Aug 20 20:36:30 2011
SUCCESS: diskgroup DATA was dismounted

ASM disk permissions:

[oracle_at_rac02b ~]$ ls -l /dev/asm-disk*

brw-rw---- 1 grid asmadmin 8,  17 Aug 20 20:37 /dev/asm-disk1
brw-rw---- 1 grid asmadmin 8, 161 Aug 20 16:55 /dev/asm-disk10
brw-rw---- 1 grid asmadmin 8, 177 Aug 20 20:29 /dev/asm-disk11
brw-rw---- 1 grid asmadmin 8,  33 Aug 20 20:37 /dev/asm-disk2
brw-rw---- 1 grid asmadmin 8,  49 Aug 20 20:37 /dev/asm-disk3
brw-rw---- 1 grid asmadmin 8,  65 Aug 20 20:36 /dev/asm-disk4
brw-rw---- 1 grid asmadmin 8,  81 Aug 20 20:36 /dev/asm-disk5
brw-rw---- 1 grid asmadmin 8,  97 Aug 20 20:36 /dev/asm-disk6
brw-rw---- 1 grid asmadmin 8, 113 Aug 20 20:36 /dev/asm-disk7
brw-rw---- 1 grid asmadmin 8, 129 Aug 20 16:55 /dev/asm-disk8
brw-rw---- 1 grid asmadmin 8, 145 Aug 20 20:29 /dev/asm-disk9

Weird, everything looks ok so far. The error message made me try to add oracle to the asmadmin group though. This is not necessary to my knowledge and as far as I remember I never did that for a RAC before. But it helped.

[root_at_rac02b ~]# usermod -a -G asmadmin oracle

[oracle_at_rac02b ~]$ id
uid=1101(oracle) gid=1000(oinstall)
groups=1000(oinstall),1020(asmadmin),1021(asmdba),1031(dba),1032(oper)

[oracle_at_rac02b ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Sat Aug 20 20:49:14 2011

Connected to an idle instance.

SQL> startup nomount
ORACLE instance started.

Total System Global Area 626327552 bytes

Fixed Size                  2229080 bytes
Variable Size             184552616 bytes
Database Buffers          436207616 bytes
Redo Buffers                3338240 bytes

SQL> CREATE DATABASE DS0P
2 USER SYS IDENTIFIED BY manager1
USER SYSTEM IDENTIFIED BY manager1
EXTENT MANAGEMENT LOCAL
DEFAULT TEMPORARY TABLESPACE TEMP
UNDO TABLESPACE UNDOTBS1
DEFAULT TABLESPACE USERS
CHARACTER SET WE8ISO8859P15
NATIONAL CHARACTER SET AL16UTF16
ARCHIVELOG
FORCE LOGGING; 3 4 5 6 7 8 9 10 11 Database created.

I did another experiment before that one which I unfortunately did not document. There I had another workaround. It seemed to work okay once I had created a database via dbca. I don't know what it does to fix the issue until know. Will let you know if I get to investigate it further.

For the moment:

Adding oracle to asmadmin seemed to fix it at least for me.

Best regards,
Robert
http://robertvsoracle.blogspot.com

On Sat, Aug 20, 2011 at 12:14 AM, Li Li <litanli_at_gmail.com> wrote:
> Hi, List,
>
> Has anyone on the list applied the July CPU? We just installed it on
> one of our non-production RAC nodes and now we can't even do a
> 'startup nomount' successfully. Logs show the following error:
>
> ORACLE_BASE from environment = /opt/app/oracle
> Errors in file /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_rbal_25712.trc:
> ORA-15183: ASMLIB initialization error [driver/agent not installed]
> WARNING: FAILED to load library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so
> Errors in file /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_rbal_25712.trc:
> ORA-15183: ASMLIB initialization error [driver/agent not installed]
> SUCCESS: diskgroup HR_DATA was mounted
> NOTE: dependency between database HRDB and diskgroup resource
> ora.HR_DATA.dg is established
> Errors in file /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_ora_25586.trc
> �(incident=7209):
> ORA-00600: internal error code, arguments: [kfioTranslateIO03], [],
> [], [], [], [], [], [], [], [], [], []
> Incident details in:
> /opt/app/oracle/diag/rdbms/hrdb/HRDB2/incident/incdir_7209/HRDB2_ora_25586_i7209.trc
> Use ADRCI or Support Workbench to package the incident.
> See Note 411.1 at My Oracle Support for error and packaging details.
> ERROR: unrecoverable error ORA-600 raised in ASM I/O path; terminating
> process 25586
> Dumping diagnostic data in directory=[cdmp_20110819164633], requested
> by (instance=2, osid=25586), summary=[incident=7209].
>
> I tried to google and can't find much information regarding this and
> CPU. Any comments/suggestions?
>
> thanks,
> -Li
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>

--
http://www.freelists.org/webpage/oracle-l

Received on Sat Aug 20 2011 - 14:02:07 CDT

This message: [ Message body ]
Next message: Guenadi Jilevski: "Re: July CPU on 11.2.0.2"
Previous message: Thomas Roach: "Re: July CPU on 11.2.0.2"
In reply to: Li Li: "July CPU on 11.2.0.2"
Next in thread: Guenadi Jilevski: "Re: July CPU on 11.2.0.2"
Reply: Guenadi Jilevski: "Re: July CPU on 11.2.0.2"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

Original text of this message