Re: Database Will Not Mount: ORA-00205

From: David Barbour <david.barbour1_at_gmail.com>
Date: Tue, 13 Feb 2018 16:57:32 -0600
Message-ID: <CAFH+iffMf3Gf3uub+ea7ybv-Pkt26uUm7Eo01072uitXuJz96Q_at_mail.gmail.com>



Okay - got it working. Not sure why it stopped in the first place but applying the 12.1.0.2.170117 DB BP was obviously not the answer. However when I applied the Database subset patches from 12.1.0.2.180116 the disks were finally able to be 'seen' by the database.

Thanks to all for your suggestions.

On Sun, Feb 11, 2018 at 3:11 PM, David Barbour <david.barbour1_at_gmail.com> wrote:

> Not that I could see. I looked at the trace files and fed in some
> messages (as well as trying Google) but didn't find anything I thought
> might be useful.
>
> What the asmb trace file shows is:
>
> *** 2018-02-10 21:22:24.245
> NOTE: ASMB mounting group 1 (DATA)
> 2018-02-10 21:22:24.245962 :kfnbGrpSaveDisks(): saving 4 disks for grp 1
> reqid=0 flags=0x92 dvop=0
> NOTE: ASM background process initiating disk discovery for grp 1 (reqid:0)
> WARNING: group 1 (DATA) has missing disks
> ORA-15040: diskgroup is incomplete
> [disk number 0, timestamp 33019794.1680322560]
> [disk number 1, timestamp 33019794.1710338048]
> [disk number 2, timestamp 33019794.1721281536]
> [disk number 3, timestamp 33019794.1734444032]
>
> But these ARE all the disks.
>
> The disk permissions look good:
>
> [root_at_node1 mapper]# ls -al DATA*
> lrwxrwxrwx. 1 root root 8 Feb 11 14:31 DATA01 -> ../dm-15
> lrwxrwxrwx. 1 root root 8 Feb 11 14:39 DATA02 -> ../dm-11
> lrwxrwxrwx. 1 root root 8 Feb 11 14:31 DATA03 -> ../dm-19
> lrwxrwxrwx. 1 root root 7 Feb 11 14:31 DATA04 -> ../dm-7
> [root_at_node1 mapper]# ls -al ../dm-15
> brw-rw----. 1 grid asmadmin 253, 15 Feb 11 14:51 ../dm-15
> [root_at_node1 mapper]# ls -al ../dm-11
> brw-rw----. 1 grid asmadmin 253, 11 Feb 11 14:39 ../dm-11
> [root_at_node1 mapper]# ls -al ../dm-19
> brw-rw----. 1 grid asmadmin 253, 19 Feb 11 14:31 ../dm-19
> [root_at_node1 mapper]# ls -al ../dm-7
> brw-rw----. 1 grid asmadmin 253, 7 Feb 11 14:31 ../dm-7
>
> I've stopped and started multipathd to no effect.
>
>
>
> On Sun, Feb 11, 2018 at 2:15 PM, Andrew Kerber <andrew.kerber_at_gmail.com>
> wrote:
>
>> Did MOS have any suggestions?
>>
>> On Sun, Feb 11, 2018 at 10:38 AM David Barbour <david.barbour1_at_gmail.com>
>> wrote:
>>
>>> Both DBs have (some version?) of the patch installed. The one that
>>> doesn't mount (orcl) has the recommended patchset bundle installed - "DATABASE
>>> BUNDLE PATCH: 12.1.0.2.170418 (25397136)" . The one that does mount
>>> (PRDREP) indicates the patchset is the critical patch update with the same
>>> identifier - "OCW Patch Set Update : 12.1.0.2.170418 (25363740)"
>>>
>>> orcl1_at_node1:/u01/app/oracle/product/12.1.0/dbhome_1 $ opatch lsinventory
>>> Oracle Interim Patch Installer version 12.2.0.1.8
>>> Copyright (c) 2018, Oracle Corporation. All rights reserved.
>>>
>>>
>>> Oracle Home : /u01/app/oracle/product/12.1.0/dbhome_1
>>> Central Inventory : /u01/app/oraInventory
>>> from : /u01/app/oracle/product/12.1.0/dbhome_1/oraInst.loc
>>> OPatch version : 12.2.0.1.8
>>> OUI version : 12.1.0.2.0
>>> Log file location : /u01/app/oracle/product/12.1.0
>>> /dbhome_1/cfgtoollogs/opatch/o
>>> patch2018-02-11_10-28-28AM_1.log
>>>
>>> Lsinventory Output file location : /u01/app/oracle/product/12.1.0/dbhome_1/cfgto
>>>
>>> ollogs/opatch/lsinv/lsinventory2018-02-11_10-28-28AM.txt
>>>
>>> ------------------------------------------------------------
>>> --------------------
>>> Local Machine Information::
>>> Hostname: node1.istation.com
>>> ARU platform id: 226
>>> ARU platform description:: Linux x86-64
>>>
>>> Installed Top-level Products (1):
>>>
>>> Oracle Database 12c
>>> 12.1.0.2.0
>>> There are 1 products installed in this Oracle Home.
>>>
>>>
>>> Interim patches (2) :
>>>
>>> Patch 25397136 : applied on Fri Feb 09 17:51:30 CST 2018
>>> Unique Patch ID: 21145057
>>> Patch description: "DATABASE BUNDLE PATCH: 12.1.0.2.170418 (25397136)"
>>>
>>> PRDREP1_at_node1:/u01/app/oracle/product/12.1.0/dbhome_2 $ opatch
>>> lsinventory
>>> Oracle Interim Patch Installer version 12.2.0.1.8
>>> Copyright (c) 2018, Oracle Corporation. All rights reserved.
>>>
>>>
>>> Oracle Home : /u01/app/oracle/product/12.1.0/dbhome_2
>>> Central Inventory : /u01/app/oraInventory
>>> from : /u01/app/oracle/product/12.1.0/dbhome_2/oraInst.loc
>>> OPatch version : 12.2.0.1.8
>>> OUI version : 12.1.0.2.0
>>> Log file location : /u01/app/oracle/product/12.1.0
>>> /dbhome_2/cfgtoollogs/opatch/opatch2018-02-11_10-30-25AM_1.log
>>>
>>> Lsinventory Output file location : /u01/app/oracle/product/12.1.0
>>> /dbhome_2/cfgtoollogs/opatch/lsinv/lsinventory2018-02-11_10-30-25AM.txt
>>>
>>> ------------------------------------------------------------
>>> --------------------
>>> Local Machine Information::
>>> Hostname: node1.istation.com
>>> ARU platform id: 226
>>> ARU platform description:: Linux x86-64
>>>
>>> Installed Top-level Products (1):
>>>
>>> Oracle Database 12c
>>> 12.1.0.2.0
>>> There are 1 products installed in this Oracle Home.
>>>
>>>
>>> Interim patches (2) :
>>>
>>> Patch 25363740 : applied on Mon Jul 10 11:35:49 CDT 2017
>>> Unique Patch ID: 20990699
>>> Patch description: "OCW Patch Set Update : 12.1.0.2.170418 (25363740)"
>>>
>>>
>>>
>>> On Sun, Feb 11, 2018 at 9:45 AM, Chris Taylor <
>>> christopherdtaylor1994_at_gmail.com> wrote:
>>>
>>>> David,
>>>>
>>>> Does this DB run out of a different DB_HOME (not GI_HOME) versus the
>>>> other db that is working?
>>>>
>>>> *IF* it runs out of a different DB_HOME, verify that patch is installed
>>>> to that home - maybe that home is missing the patch you mentioned?
>>>>
>>>> + Apply 12.1.0.2.170117 DB BP or above on RDBMS_HOME ( Doc ID 2247077.1
>>>> )
>>>>
>>>> Chris
>>>>
>>>>
>>>> On Sat, Feb 10, 2018 at 9:38 PM, David Barbour <
>>>> david.barbour1_at_gmail.com> wrote:
>>>>
>>>>> What is interesting is that we have another 12.1.0.2 database on this
>>>>> cluster and it's not having any issues.
>>>>>
>>>>> The control files are on ASM:
>>>>>
>>>>> SQL> show parameter control_files;
>>>>>
>>>>> NAME TYPE VALUE
>>>>> ------------------------------------ -----------
>>>>> ------------------------------
>>>>> control_files string
>>>>> +DATA/ORCL/CONTROLFILE/current
>>>>> .263.881061111,
>>>>> +DATA/ORCL/CON
>>>>>
>>>>> TROLFILE/current.270.881061111
>>>>>
>>>>> The database alert log shows the following:
>>>>>
>>>>> NOTE: ASMB mounting group 1 (DATA)
>>>>> NOTE: ASM background process initiating disk discovery for grp 1
>>>>> (reqid:0)
>>>>> WARNING: group 1 (DATA) has missing disks
>>>>> ORA-15040: diskgroup is incomplete
>>>>> WARNING: group 1 is being dismounted.
>>>>> WARNING: ASMB force dismounting group 1 (DATA) due to missing disks
>>>>> SUCCESS: diskgroup DATA was dismounted
>>>>>
>>>>> The database shows the diskgroup and associated disks are mounted and
>>>>> present:
>>>>>
>>>>> 1 select a.disk_number, a.name, b.name, b.state from v$asm_disk a
>>>>> 2 join v$asm_diskgroup b on a.group_number=b.group_number
>>>>> 3* order by b.name, a.disk_number
>>>>> SQL> /
>>>>>
>>>>> DISK_NUMBER NAME NAME
>>>>> STATE
>>>>> ----------- ------------------------------
>>>>> ------------------------------ -----------
>>>>> 0 DATA_0000 DATA
>>>>> MOUNTED
>>>>> 1 DATA_0001 DATA
>>>>> MOUNTED
>>>>> 2 DATA_0002 DATA
>>>>> MOUNTED
>>>>> 3 DATA_0003 DATA
>>>>> MOUNTED
>>>>> 0 FRA_0000 FRA
>>>>> MOUNTED
>>>>> 1 FRA_0001 FRA
>>>>> MOUNTED
>>>>> 0 GRID_0000 GRID
>>>>> MOUNTED
>>>>> 1 GRID_0001 GRID
>>>>> MOUNTED
>>>>> 2 GRID_0002 GRID
>>>>> MOUNTED
>>>>> 3 GRID_0003 GRID
>>>>> MOUNTED
>>>>> 0 PRDREPDATA01_0000 PRDREPDATA01
>>>>> MOUNTED
>>>>>
>>>>> DISK_NUMBER NAME NAME
>>>>> STATE
>>>>> ----------- ------------------------------
>>>>> ------------------------------ -----------
>>>>> 1 PRDREPDATA01_0001 PRDREPDATA01
>>>>> MOUNTED
>>>>> 2 PRDREPDATA01_0002 PRDREPDATA01
>>>>> MOUNTED
>>>>> 3 PRDREPDATA01_0003 PRDREPDATA01
>>>>> MOUNTED
>>>>> 0 PRDREPFRA_0000 PRDREPFRA
>>>>> MOUNTED
>>>>> 0 REDO1_0000 REDO1
>>>>> MOUNTED
>>>>> 0 REDO2_0000 REDO2
>>>>> MOUNTED
>>>>>
>>>>> 17 rows selected.
>>>>>
>>>>> The ASM alert log shows:
>>>>>
>>>>> NOTE: client orcl1:orcl:scan01-int no longer has group 1 (DATA) mounted
>>>>>
>>>>> CRS shows:
>>>>>
>>>>> [grid_at_node1 ~]$ crsctl stat res -t
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>> Name Target State Server State
>>>>> details
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>> Local Resources
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>> ora.ASMNET1LSNR_ASM.lsnr
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.DATA.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.FRA.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.GRID.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.LISTENER.lsnr
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.PRDREPDATA01.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.PRDREPFRA.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.REDO1.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.REDO2.dg
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.net1.network
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.ons
>>>>> ONLINE ONLINE node1 STABLE
>>>>> ora.proxy_advm
>>>>> OFFLINE OFFLINE node1 STABLE
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>> Cluster Resources
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>> ora.LISTENER_SCAN1.lsnr
>>>>> 1 ONLINE ONLINE node1 STABLE
>>>>> ora.LISTENER_SCAN2.lsnr
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.LISTENER_SCAN3.lsnr
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.MGMTLSNR
>>>>> 1 ONLINE ONLINE node1
>>>>> 169.254.112.28 1.1.1
>>>>> .1,STABLE
>>>>> ora.asm
>>>>> 1 ONLINE ONLINE node1
>>>>> Started,STABLE
>>>>> 2 ONLINE OFFLINE STABLE
>>>>> ora.cvu
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.mgmtdb
>>>>> 1 ONLINE ONLINE node1
>>>>> Open,STABLE
>>>>> ora.node1.vip
>>>>> 1 ONLINE ONLINE node1 STABLE
>>>>> ora.node2.vip
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.orcl.db
>>>>> 1 ONLINE OFFLINE STABLE
>>>>> 2 OFFLINE OFFLINE STABLE
>>>>> ora.orcl.test.svc
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.prdrep.db
>>>>> 1 ONLINE ONLINE node1
>>>>> Open,HOME=/u01/app/o
>>>>>
>>>>> racle/product/12.1.0
>>>>>
>>>>> /dbhome_2,STABLE
>>>>> 2 ONLINE OFFLINE STABLE
>>>>> ora.prdrep.prdrepsvc.svc
>>>>> 1 ONLINE ONLINE node1 STABLE
>>>>> 2 ONLINE OFFLINE STABLE
>>>>> ora.qosmserver
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.scan1.vip
>>>>> 1 ONLINE ONLINE node1 STABLE
>>>>> ora.scan2.vip
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ora.scan3.vip
>>>>> 1 OFFLINE OFFLINE STABLE
>>>>> ------------------------------------------------------------
>>>>> --------------------
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Feb 10, 2018 at 7:14 PM, Vishnu <vishnukumarmp_at_gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi David, please check whether the file system which has the
>>>>>> database control file is mounted and able to read ? check your asm alert
>>>>>> log to get additional info.
>>>>>>
>>>>>> $crsctl stat res -t
>>>>>>
>>>>>> Above command will provide online/offline status of all of your
>>>>>> resources including asm disk groups.
>>>>>>
>>>>>> Thanks,
>>>>>> Vishnu
>>>>>>
>>>>>> On Sat, Feb 10, 2018 at 1:38 PM, David Barbour <
>>>>>> david.barbour1_at_gmail.com> wrote:
>>>>>>
>>>>>>> Good Afternoon,
>>>>>>>
>>>>>>> I have upgraded the GI to 12.2 and the 12.1 databases are failing to
>>>>>>> start with the error:
>>>>>>>
>>>>>>> ORA-00205: error in identifying control file, check alert log for
>>>>>>> more info
>>>>>>>
>>>>>>> This behavior is described in Doc ID 2247077.1 but the solution is
>>>>>>> to apply 12.1.0.2.170117 DB BP or above on RDBMS_HOME.
>>>>>>> Unfortunately this doesn't seem to be the answer in this case as the
>>>>>>> database is already at 12.1.0.2.170418.
>>>>>>>
>>>>>>> CRS shows the following:
>>>>>>>
>>>>>>> ora.orcl.db
>>>>>>> 1 ONLINE OFFLINE STABLE
>>>>>>> 2 OFFLINE OFFLINE STABLE
>>>>>>>
>>>>>>> I can't seem to get this database to go offline:offline.
>>>>>>>
>>>>>>> Has anyone had a similar issue?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> --
>> Andrew W. Kerber
>>
>> 'If at first you dont succeed, dont take up skydiving.'
>>
>
>

--
http://www.freelists.org/webpage/oracle-l
Received on Tue Feb 13 2018 - 23:57:32 CET

Original text of this message