Re: strange error on switchover to standby

Oh yeah, oracle 12c/Linux x86_64/ 3 node rac/ asm.

On Fri, Jan 20, 2017 at 4:50 PM, Andrew Kerber <andrew.kerber_at_gmail.com> wrote:

> So here is where I am. On the primary side, lets call it testp,
> switchover works fine using dgmgrl. (switchover to testd).
>
> I go over to the testd side, and run a switchover command in testd
> (switchover to testp), I get the error message:
>
> Error: ORA-48189: OS command to create directory failed
> Error: ORA-16625: cannot reach database "TESTD"
>
> However, if I go back over to the original primary side, and run the same
> switchover command there, it works fine.
>
>
> I am sure its some sort of privilege problem, but even with the trace
> level set to support, it wont tell me what command is failing. And
> unfortunately, I cant even tell from the error message whether its on the
> testp or testd side that it is trying to create a file, though it always
> seem to be when its communicating to testd.
>
> There is no command in the drc*.log or the alert log. Is there anywhere
> else I could look?
>
>
>
>
> On Fri, Jan 20, 2017 at 3:38 PM, Andrew Kerber <andrew.kerber_at_gmail.com>
> wrote:
>
>> I tried that one already. It simply wont show the command that is
>> failing:
>> Properties:
>> FastStartFailoverThreshold = '30'
>> OperationTimeout = '30'
>> TraceLevel = 'SUPPORT'
>> FastStartFailoverLagLimit = '30'
>> CommunicationTimeout = '180'
>> ObserverReconnect = '0'
>> FastStartFailoverAutoReinstate = 'TRUE'
>> FastStartFailoverPmyShutdown = 'TRUE'
>> BystandersFollowRoleChange = 'ALL'
>> ObserverOverride = 'FALSE'
>> ExternalDestination1 = ''
>> ExternalDestination2 = ''
>> PrimaryLostWriteAction = 'CONTINUE'
>>
>>
>> On Fri, Jan 20, 2017 at 2:47 PM, Bernhard de Cock Buning - GRID-IT <
>> bdcbuning_at_grid-it.nl> wrote:
>>
>>> Hi Andrew,
>>>
>>> Would expect to see more details in the drc logfile.
>>>
>>> If you find out it is related to one server(s) cluster, It starts to
>>> look like it is related to the log/trace directories.
>>> Have a look at the ADR on that host and the permissions, maybe it is
>>> related to the diag_dest parameter on the standby location where it defines
>>> an incorrect location. But I would in that case expect you would already
>>> have issues for some time and not only when you perform a switchover.
>>>
>>> But if you need more tracing info you can execute in dgmgrll (assume
>>> you use 11.2.0.3 or higher)
>>> Admin is the default tracelevel, change it to support.
>>>
>>> edit configuration set property tracelevel=support
>>>
>>>
>>> Hope this helps
>>>
>>> Greetings,
>>> Bernhard
>>>
>>> Op 20 jan. 2017, om 21:32 heeft Andrew Kerber <andrew.kerber_at_gmail.com>
>>> het volgende geschreven:
>>>
>>> interestingly, if I moved over to the original primary servers to run
>>> the command, the switchover succeeds. It only fails with the error on one
>>> cluster, which means I at least know for sure what servers the error is
>>> coming from. Now if I could just figure out what command.
>>>
>>> On Fri, Jan 20, 2017 at 2:10 PM, Andrew Kerber <andrew.kerber_at_gmail.com>
>>> wrote:
>>>
>>>> I was trailing that file, and it does not contain the command that
>>>> failed. Here is what is in the broker log file:
>>>>
>>>> Failed to connect to remote database TEST. Error is ORA-48189
>>>> Failed to send message to site TEST. Error code is ORA-48189.
>>>> database TEST unable to contact primary database for version check;
>>>> status ORA-48189
>>>> completing bootstrap of this database
>>>> Notifying Oracle Clusterware to buildup
>>>>
>>>>
>>>> On Fri, Jan 20, 2017 at 2:05 PM, Bernhard de Cock Buning - GRID-IT <
>>>> bdcbuning_at_grid-it.nl> wrote:
>>>>
>>>>> Hi Andrew,
>>>>>
>>>>> Check out the broker logfile, when correct this will explain where it
>>>>> goes wrong.
>>>>> You can find this logfile drc*.log in the same location as the
>>>>> alert.log of the database.
>>>>> Be sure you check both location (the primary and standby)
>>>>>
>>>>> You can also enable additional tracing in the broker but as mentioned
>>>>> I believe the drc log will contain the error.
>>>>>
>>>>> Greetings,
>>>>> Bernhard
>>>>>
>>>>> Op 20 jan. 2017, om 20:56 heeft Andrew Kerber <andrew.kerber_at_gmail.com>
>>>>> het volgende geschreven:
>>>>>
>>>>> I am getting a strange error when I am trying to switchover to my
>>>>> standby database. The first switchover worked fine, so I am switching over
>>>>> to my original primary, and I am getting this message:
>>>>>
>>>>> DGMGRL> switchover to 'TEST';
>>>>> Performing switchover NOW, please wait...
>>>>> Operation requires a connection to instance "TEST3" on database "TEST"
>>>>> Connecting to instance "TEST3"...
>>>>> Connected as SYSDBA.
>>>>> Error: ORA-48189: OS command to create directory failed
>>>>> Error: ORA-16625: cannot reach database "TEST"
>>>>>
>>>>>
>>>>> It looks like it is trying to create a directory, but I cannot figure
>>>>> out what directory it is trying to creat. I expect its a simple ownership
>>>>> problem, but I cannot figure out what directory I need to work on. I tried
>>>>> setting the trace level to support in dgmgrl, but it didnt show the
>>>>> information I needed. Any ideas?
>>>>>
>>>>>
>>>>> --
>>>>> Andrew W. Kerber
>>>>>
>>>>> 'If at first you dont succeed, dont take up skydiving.'
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Andrew W. Kerber
>>>>
>>>> 'If at first you dont succeed, dont take up skydiving.'
>>>>
>>>
>>>
>>>
>>> --
>>> Andrew W. Kerber
>>>
>>> 'If at first you dont succeed, dont take up skydiving.'
>>>
>>>
>>>
>>
>>
>> --
>> Andrew W. Kerber
>>
>> 'If at first you dont succeed, dont take up skydiving.'
>>
>
>
>
> --
> Andrew W. Kerber
>
> 'If at first you dont succeed, dont take up skydiving.'
>

-- 
Andrew W. Kerber

'If at first you dont succeed, dont take up skydiving.'

--
http://www.freelists.org/webpage/oracle-l