RE: pr00 hanging in "parallel recovery control message reply"

From: Noveljic Nenad <nenad.noveljic_at_vontobel.com>
Date: Thu, 15 Oct 2020 17:42:01 +0000
Message-ID: <31818_1602783738_5F8889FA_31818_17771_1_8c9e4876267e41c1ae8f272c5f1e7257_at_vontobel.com>



Von: Noveljic Nenad <nenad.noveljic_at_vontobel.com<mailto:nenad.noveljic_at_vontobel.com>> Datum Donnerstag, 15. Okt. 2020, 7:21 PM An: Gogala, Mladen (Parsippany) <mladen.gogala_at_Fiserv.com<mailto:mladen.gogala_at_Fiserv.com>> Betreff: RE: pr00 hanging in "parallel recovery control message reply"

Hi Mladen,

The exact version is 19.7.0.0.200414 .

The restarted recovery works well.

The restore was a part of our backup verification process which basically runs an infinite loop to restore all production databases. I was wondering what additional useful information I might gather should the problem reappear. System dump is of little use, I’m afraid, as it would only show that the coordinator is waiting on the recovery parallel processes and vice versa.

Best regards,

Nenad

From: Gogala, Mladen (Parsippany) <mladen.gogala_at_Fiserv.com> Sent: Donnerstag, 15. Oktober 2020 18:38 To: Noveljic Nenad <nenad.noveljic_at_vontobel.com> Subject: RE: pr00 hanging in "parallel recovery control message reply"

Hi Nenad!
You should take a system dump and upload it to support. Goes without saying, please open a SR. This is a program error, a recovery component is waiting for the message which never came. If the problem is urgent, about the only thing you can do is to cancel the recovery, try again and hope for the best. Maybe, the component that should have sent a "parallel recovery control message reply" will do so the next time you try. For my personal curiosity: what is the upgrade bundle on that DB? Is it 19.8? Regards

From: oracle-l-bounce_at_freelists.org<mailto:oracle-l-bounce_at_freelists.org> <oracle-l-bounce_at_freelists.org<mailto:oracle-l-bounce_at_freelists.org>> On Behalf Of Noveljic Nenad Sent: Thursday, October 15, 2020 5:17 AM To: ORACLE-L (oracle-l_at_freelists.org<mailto:oracle-l_at_freelists.org>) <oracle-l_at_freelists.org<mailto:oracle-l_at_freelists.org>> Subject: pr00 hanging in "parallel recovery control message reply"

  ⚠ EXTERNAL MESSAGE – Think Before You Click

A 19c database recovery has been hanging for days:

2020-10-12T11:38:38.418894+02:00
Media Recovery Log /u00/oracle/oradata06/DB/arch/DB_1_155811_1016535193.arc

pr00 has been hanging in

SQL> select event,state,seconds_in_wait from v$session where program like '%PR00%' ;

EVENT                                                            STATE               SECONDS_IN_WAIT
---------------------------------------------------------------- ------------------- ---------------
parallel recovery control message reply                          WAITING                      257183


The other 96 parallel recovery processes have been waiting in “parallel recovery slave next change”:

SQL> select event,state,count(*) from v$session where program like '%PR%' and program not like '%PR00%' and status='ACTIVE' group by event,state;

EVENT                                                            STATE                 COUNT(*)
---------------------------------------------------------------- ------------------- ----------
parallel recovery slave next change                              WAITING                     96


It seems that the synchronization between parallel recovery processes went astray.

Does anybody have any idea how to troubleshoot this?

Thanks,

Nenad

http://nenadnoveljic.com/blog/<https://urldefense.proofpoint.com/v2/url?u=http-3A__nenadnoveljic.com_blog_&d=DwMFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=PFGMmv_4ysqEQpf_uV6FERWGdUNscZVcotFkX33Eups&m=f7ALDa5pBngmZXyMdDgRvLi6s0Oq62n8XGPAEnDeb80&s=VDei194kvpd15fvMRqvtu5IE108WRhl2U13FRmEAp6U&e=>



Please consider the environment before printing this e-mail. Bitte denken Sie an die Umwelt, bevor Sie dieses E-Mail drucken.

Important Notice

This message is intended only for the individual named. It may contain confidential or privileged information. If you are not the named addressee you should in particular not disseminate, distribute, modify or copy this e-mail. Please notify the sender immediately by e-mail, if you have received this message by mistake and delete it from your system. Without prejudice to any contractual agreements between you and us which shall prevail in any case, we take it as your authorization to correspond with you by e-mail if you send us messages by e-mail. However, we reserve the right not to execute orders and instructions transmitted by e-mail at any time and without further explanation. E-mail transmission may not be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete. Also processing of incoming e-mails cannot be guaranteed. All liability of Vontobel Holding Ltd. and any of its affiliates (hereinafter collectively referred to as "Vontobel Group") for any damages resulting from e-mail use is excluded. You are advised that urgent and time sensitive messages should not be sent by e-mail and if verification is required please request a printed version. Please note that all e-mail communications to and from the Vontobel Group are subject to electronic storage and review by Vontobel Group. Unless stated to the contrary and without prejudice to any contractual agreements between you and Vontobel Group which shall prevail in any case, e-mail-communication is for informational purposes only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. The legal basis for the processing of your personal data is the legitimate interest to develop a commercial relationship with you, as well as your consent to forward you commercial communications. You can exercise, at any time and under the terms established under current regulation, your rights. If you prefer not to receive any further communications, please contact your client relationship manager if you are a client of Vontobel Group or notify the sender. Please note for an exact reference to the affected group entity the corporate e-mail signature. For further information about data privacy at Vontobel Group please consult www.vontobel.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.vontobel.com&d=DwMFAg&c=rE3mhBYFJfJGqQ7WI0-DPw&r=PFGMmv_4ysqEQpf_uV6FERWGdUNscZVcotFkX33Eups&m=f7ALDa5pBngmZXyMdDgRvLi6s0Oq62n8XGPAEnDeb80&s=QETq_E1-XpgH9QHJurwzbRsoP9j29x3LRaEVwHqpbIc&e=>.

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css">p { font-family: Arial;font-size:9pt }</style>
</head>
<body>
<p>
<br>Important Notice</br>
<br />

This message is intended only for the individual named. It may contain confidential or privileged information. If you are not the named addressee you should in particular not disseminate, distribute, modify or copy this e-mail. Please notify the sender immediately by e-mail, if you have received this message by mistake and delete it from your system.<br /> Without prejudice to any contractual agreements between you and us which shall prevail in any case, we take it as your authorization to correspond with you by e-mail if you send us messages by e-mail. However, we reserve the right not to execute orders and instructions transmitted by e-mail at any time and without further explanation.<br /> E-mail transmission may not be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete. Also processing of incoming e-mails cannot be guaranteed. All liability of Vontobel Holding Ltd. and any of its affiliates (hereinafter collectively referred to as "Vontobel Group") for any damages resulting from e-mail use is excluded. You are advised that urgent and time sensitive messages should not be sent by e-mail and if verification is required please request a printed version.</br> Please note that all e-mail communications to and from the Vontobel Group are subject to electronic storage and review by Vontobel Group. Unless stated to the contrary and without prejudice to any contractual agreements between you and Vontobel Group which shall prevail in any case, e-mail-communication is for informational purposes only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction.<br /> The legal basis for the processing of your personal data is the legitimate interest to develop a commercial relationship with you, as well as your consent to forward you commercial communications. You can exercise, at any time and under the terms established under current regulation, your rights. If you prefer not to receive any further communications, please contact your client relationship manager if you are a client of Vontobel Group or notify the sender. Please note for an exact reference to the affected group entity the corporate e-mail signature. For further information about data privacy at Vontobel Group please consult <a href="https://www.vontobel.com">www.vontobel.com</a>.<br />
</p>
</body>
</html>

--
http://www.freelists.org/webpage/oracle-l
Received on Thu Oct 15 2020 - 19:42:01 CEST

Original text of this message