Home » Infrastructure » Windows » Oracle 9i DB Closes Randomly (Oracle DB 9.2.0.1, Windows 2003 Server R2, VMWare ESXi)
Oracle 9i DB Closes Randomly [message #360147] Wed, 19 November 2008 15:55 Go to next message
mainfr4me
Messages: 5
Registered: November 2008
Location: Waukesha, WI
Junior Member
Hello,
I've been doing some searching for a while, and have yet to find a good reasoning behind the behavior I'm seeing (and repeated now 2x).

I have a database running on a Windows 2003 Server R2 VMware system, running Oracle 9i (9.2.0.1) version database.

I have a number of these systems out at other locations and customers without hassle, however, I have this one which I cannot keep a system running properly for more than a month or two. I have not seen any new Group Policy items propagate down to the server that would affect it, the site has not installed any additional software that would cause problems, however after a random amount of time and users connected (sometimes happens in the middle of the night, middle of the day, etc), the database goes into a stopped state. I can open the database back up with no errors or trouble, however the database will eventually go back into a closed state.

Has anyone else run into this? I'm not a trained Oracle DB, and I'm doing my best off what I can find on-line (the company decided not to hire another DBA after our last one left), so that's why I've come to ask, HELP?!?!

As a reward, whoever gives me the magical answer will get cookies (and I'm dead serious, I'll mail you cookies this has me tearin my hair out, but I'll make sure none of it makes it into the cookies).
Re: Oracle 9i DB Closes Randomly [message #360154 is a reply to message #360147] Wed, 19 November 2008 19:06 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
Do you see any errors in the alert log and/or other errors generated in the cdump/udump/bdump directories ?

Does the Windows event viewer indicate any errors ?

Are ALL of these systems identical (ie. same O/S and Oracle patchset level, hardware) ?

Unfortunately, you may have a hard time getting support from Oracle directly since Oracle isn't supported on VMWare.
Re: Oracle 9i DB Closes Randomly [message #360335 is a reply to message #360154] Thu, 20 November 2008 08:08 Go to previous messageGo to next message
mainfr4me
Messages: 5
Registered: November 2008
Location: Waukesha, WI
Junior Member
From what I can see, the only entries into the Windows Event Viewer an items stating that the database processes start up, and then nothing other than the normal client connection entries up until the db closes.

I will double check the dump directories and see what I can find in a few minutes.

The systems that these run on are pretty similar, I can't say identical for the Server 2003 systems because more of our sites run on simple XP Pro systems.

Yea, Oracle directly has not been a ball of laughs trying to get help. Although, I have about a dozen of these similar configs running at other sites, so if it would have happened to another site, I would think it would have happened to the others already.
Re: Oracle 9i DB Closes Randomly [message #360359 is a reply to message #360147] Thu, 20 November 2008 11:17 Go to previous messageGo to next message
mainfr4me
Messages: 5
Registered: November 2008
Location: Waukesha, WI
Junior Member
Alright, good news.

I logged in and found the bdump directory, and I see Saturday when the system initially went offline, there is an entry stating that the system could not write to one of the db files, so it closed to prevent corruption.

What I'm talking to the site now is if they have a backup running or something quirky going on that might have that happen. It reminds me of some of the errors I'd see when a HD would go bad in some of our older systems (not the same error, but similar behavior). I know they use VMotion for high availability, and I'm curious if it's a delayed write to the file and the engine is freaking out over it.
Re: Oracle 9i DB Closes Randomly [message #360362 is a reply to message #360359] Thu, 20 November 2008 11:42 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
Does it coincide with any migration performed with VMotion ? Any errors logged for the disks themselves or disk controller ?

If it appears to be the same file each time, it may not hurt to run dbv against the file in question.

Out of curiosity, has VMotion performed well for your operations ?

[Updated on: Thu, 20 November 2008 11:42]

Report message to a moderator

Re: Oracle 9i DB Closes Randomly [message #360363 is a reply to message #360362] Thu, 20 November 2008 11:48 Go to previous messageGo to next message
mainfr4me
Messages: 5
Registered: November 2008
Location: Waukesha, WI
Junior Member
So far I don't believe any other customers run VMotion, and I've asked the site for more info on the setup and if there were any errors/failovers/etc when this happened on the system.

I had seen something noted that for 10g databases, VMotion seemed to handle it, but during a switch could have a 10-15 second delay (which could account for the write delay on this system).

I have a call with the site's manager in about an hour, so if I find out more then, I'll post it up.
Re: Oracle 9i DB Closes Randomly [message #387526 is a reply to message #360147] Thu, 19 February 2009 15:23 Go to previous messageGo to next message
jrmtl
Messages: 27
Registered: February 2006
Junior Member
I am having the exact same problem , except i'm not on vmware and this is a physical server.

Have you pinpointed exacty what is going on ?

My database service just stops with no logs anywhere
Re: Oracle 9i DB Closes Randomly [message #387532 is a reply to message #387526] Thu, 19 February 2009 15:40 Go to previous message
mainfr4me
Messages: 5
Registered: November 2008
Location: Waukesha, WI
Junior Member
Hm - I thought I had updated this.

For this case, it turned out the site admin had decided to run backups both of the VMWare disc (which the VMWare OS handled fine), but also inside the VM's OS backup the individual DB file which would lock the DB files, so when Oracle tried to write, it would detect the file lock and fail.

I had him modify his backup procedure (we were already doing nightly backups via a scheduled task to a zip file on the network) to not backup the OS inside the VM, continue as he wanted to backup the disk files, and all was happy. Unfortunatly, this meant I still had to rebuild that instance again (had corrupted the DB files completely), but we avoided it for the future.
Previous Topic: Windows 2003 9i database service shuts down with no logs
Next Topic: Configuring Oracle 10G Express Client on win XP
Goto Forum:
  


Current Time: Sat Aug 23 06:53:21 CDT 2014

Total time taken to generate the page: 0.11676 seconds