RE: Opinion on change control for DBA scripts
Date: Fri, 28 Jun 2013 11:53:57 -0500
Message-ID: <AD8FE6616C097545A4C9A8B0792909AC290F9DE36A_at_DNBEXCH01.dnbint.net>
I'd like to expand on a topic a bit if you all don't mind. I've been wanting to ask this for some time and Sandra's unfortunate experience has pushed me to get my act together.
First with change control, my concern is mostly over keeping everything in sync. We have 20 to 30 maintenance scripts that involve a lot of shell commands (although I really liked the idea of using views to support SQL scripts) and maintain them on a NAS. We started years ago supporting just 50 database servers so that situation worked well. Now our team supports 400+ database servers, so instead of trying to determine what the limits are with NAS and having to open 350+ tickets to have the NAS attached everywhere, I went with using a consistent env variable to signify the home base where our scripts are stored, like $NAS, added that to all .bash_profile, and where NAS wasn't attached I created a local copy of our scripts. I have a script that is a wrapper around "rsync" to keep all local copies of our scripts in sync with our central location (just 1 EM server that we treat like a central repository).
All that stuff is probably what any decent change control app would do, but the current setup has morphed over time and also we've run into the situation where IT budgets tend to "dry up" by the time it comes to helping DBAs with tools.
My expansion to this topic is about job scheduling. For those who need to support hundreds of database servers, what do you use to schedule maintenance and monitoring jobs across them all? We have many jobs in EM but have also inherited systems with jobs run out of cntrl-m (which is a pain because we don't have access to the tool). In either case preferred credentials have to be maintained (which is also "fun" on all these systems with password expiration rules) and setting up proper checks and balances. For the latter I'm referring to issues with schedulers. It's making sure that what you expected to run actually did run, that key jobs didn't somehow disappear, that agents didn't suddenly crash and no one noticed, etc.
Dave Herring
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Sandra Becker
Sent: Wednesday, June 26, 2013 2:35 PM
To: oracle-l
Subject: Opinion on change control for DBA scripts
One of our VP's yelled at me this morning and slammed my door when he left my office. He was angry because I didn't check my scripts into the change control they use for the application code so wasn't able to give him information no exec has asked me for in my entire tenure with the company. I was collecting space usage stats to enable me to track for analysis and planning. A few months I lost all my scripts because IT decided it wasn't important to back them up. Hadn't gotten around to recreating this particular script since I was the only person using the information. He also asked for information on some disk space that isn't related to any of my databases and I don't have access to. That's when he yelled at me to get him what I could and slammed my door. So what do most DBAs do? Do you check your scripts into source control? have copies? don't worry about it? I have begun keeping copies on a thumb drive since the second time I lost everything.
--
Sandy
Transzap, Inc.
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
Received on Fri Jun 28 2013 - 18:53:57 CEST