Re: Protecting production from "us"
Date: Fri, 4 Dec 2015 09:46:06 +1100
Message-ID: <CAFeFPA9pMyEVNd0pMmGT3Q_kCy=T3Yq6fikmkVb8RK4D3LKp8A_at_mail.gmail.com>
Hi
Isn't the whole idea of RAC that you can shutdown one instance for whatever
reason and not be affected (much)
Or was the locking issue unrelated? (murphy's law).
Jack
Jack van Zanen
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient. If you are not the intended recipient, please be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received this e-mail in error, please contact the sender and delete all copies. Thank you for your cooperation
On Fri, Dec 4, 2015 at 9:36 AM, Tim Gorman <tim_at_evdbt.com> wrote:
> Each time I've made a mistake, I have absolutely *longed* for the freedom
> to walk away and not be responsible for fixing the mess I created, instead
> of feeling morally obligated to spend the next 30-60 hours working my tail
> off to make things right.
>
> Sounds like my kinda place, Alfredo! Where do I sign up? :)
>
> Shouldn't management lead by example? Shouldn't they also be subject to
> termination for any mistake they make?
>
>
>
>
> On 12/3/15 13:49, Alfredo Abate wrote:
>
> I'm disappointed at management's response of *the backlash is now any
> further mistakes on production will result in immediate termination. *I
> don't see how any person (in any field) could work knowing that if they
> make another mistake like this that they are terminated. Especially given
> the track record that it sounds you and the rest of your team has had
> (years between this happening). If someone was making these types of
> mistakes frequently then that is another story all together.
>
> I suppose if the system at hand cost a company tons of money for each
> outage (say a trading system or high volume eCommerce) then things might be
> a little different (maybe this is the case here).
>
> At the end of the day these machines, systems, etc are all operated by the
> all mighty error prone humans.
>
> Alfredo
>
>
>
> On Thu, Dec 3, 2015 at 2:27 PM, Alfredo Abate <alfredo.abate_at_gmail.com>
> wrote:
>
>> I like Jeremy's server side control better for the terminal background
>> colors. I'll have to look into that one.
>>
>> Thanks for that tip.
>>
>> Alfredo
>>
>>
>>
>> On Thu, Dec 3, 2015 at 1:05 PM, Jeremy Schneider <
>> <jeremy.schneider_at_ardentperf.com>jeremy.schneider_at_ardentperf.com> wrote:
>>
>>> On Thu, Dec 3, 2015 at 11:45 AM, Herring, David < <HerringD_at_dnb.com>
>>> HerringD_at_dnb.com> wrote:
>>> > · Should we look into some kind of additional controls where
>>> > commands like "srvctl stop…" cannot be run under our own accounts using
>>> > "sudo -u oracle" but instead need a different account on production?
>>> For
>>> > example, normally our unfortunate DBA would use his "scapebob" Linux
>>> account
>>> > but perhaps to perform a production shutdown he'd need to connect as
>>> > "scapebob-rw", a new, special account just for dangerous production
>>> > activities.
>>>
>>> I think that I'd be hesitant to introduce too much variation between
>>> production and test environments when it comes to processes. It's a
>>> major advantage if you can test your processes in the test tier, then
>>> run those same processes verbatim (key-for-key) in production
>>> afterwards.
>>>
>>> > · The problem in our situation was over confusion with multiple
>>> > windows. Do people set a Linux TMOUT to something short like 10 or 15
>>> > minutes, to hopefully avoid accidentally leaving production putty
>>> sessions
>>> > open?
>>>
>>> I feel like a short timeout is likely to cause more frustration in the
>>> trenches than what it's worth, for anyone who spends any significant
>>> amount of time troubleshooting production systems. Often you have
>>> multiple windows open and switch between them... an aggressive timeout
>>> really makes that much more difficult.
>>>
>>> > · Beyond changing the linux prompt and text colors (we set
>>> $PS1 with
>>> > escape sequences and various key, env-specific values) do you do
>>> anything
>>> > else for protection of production?
>>>
>>> Personally, I think background color is your best bet. Only difference
>>> from Alfredo's suggestion would be that I'd prefer having it be
>>> controlled server-side rather than relying on each engineer to setup
>>> all their terminal connections correctly. Not to mention that you
>>> could get the *wrong* bg color if it's client-side and somehow
>>> somebody ssh's between tiers.
>>>
>>> --
>>> http://about.me/jeremy_schneider
>>> --
>>> http://www.freelists.org/webpage/oracle-l
>>>
>>>
>>>
>>
>
>
-- http://www.freelists.org/webpage/oracle-lReceived on Thu Dec 03 2015 - 23:46:06 CET