| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
 |
 | |||
| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
|
| |||
Home -> Community -> Mailing Lists -> Oracle-L -> Re: Hardware / OS recommendation
OK Mogens, you're right. I agree totally with you. Commit. Respect. But.
Do I read you well that you are putting a thesis that, once proved to be true, makes it unavoidable to apply magic bullets, without proper testing? So we have to leave the scientific approach? I cannot read your post in another way, but that can be caused by my tiredness, just after a full day of final rehearsing a full-evening performance with my choir. [Jared amd list-members, I'm not trying o catch up a closed thread!]
So we end up in the oximoron, that more complexity requires more testing. Testing more complexity requires more time. Time to market becomes shorter and shorter. There is less time to test (and optimize! let's face that. Many teams of huge amounts of cheap junior java programmers think that optimization is not needed any more. Just add more hardware). We need more time to test. So, don't test, but increase complexity by adding more hardware. This rat-race must end somewhere, when we all run over the edge, like lemmings.
However, it's not only in our business that I see this happen. In Rotterdam new streetcars were put in service. Many, many problems came up, the delivered ones were sent back, new ones were temorarily refused. Now everything seems to be fine, but this is an example of 'testing in production' of renowned engineering companies. Another example: bridges (London, Rotterdam), shivering from winds during storms. Pretty mature engineering, bridgbuilding, wasn't it? Still 'not tested enough'.
This is no excuse for not testing. One should test. But, When systems are growing too complex, full testing becomes impossible. How to cope with this? I think that sufficient knowledge of the system must be available, at least in the first days/weeks/months when a system goes life. What is sufficient? Sufficient is equal to maturity and experience, of thos who have tested many isolated components, enough to be able to pinpoint a possible cause when the system fails. And then aagain, it's all a matter of money, trade-offs, common sense. A couple of years ago Mercedes-Benz decided that not the BEST product, but the product that would stand the life of the car the best for the lowest price was what they wanted. Not every site is expecting top-of-the-bill performance, avalability, consulting. We shoudl deliver the best we can, given the price the customer is willing to pay. Quality is: The expected result for the price agreed. Nothing more, nothing less. If we charge not enough, or put it another way, promise too much, it's our problem, not the customer's. We're the experts (we think), we haven to guide the customer through the process. But we should also guide the customer when he's moving targets (which they always do). Too often, just to save the relationship, we let derail the train, and plunge into the river of sadness together with the (unsatisfied) customer.
So, testing all is most likely impossible. Keeping on testing new features, in isolation, to gain knowledge still makes sense. This testing can still be done in a scientific setting. This knowledge can help solving problems in a quick (and dirty?) way, not by magic bullets, but by recognizing a symptom and know the cure. Life is pretty much an adventure. It is not possible to kwno everthing in advance. When there is no time, calculate the risk, share your thoughts with the customer, and give it a try, or not.
Don't know whether this post makes any sense anymore, bottom line is: Skyhigh complexity makes applying magic bullets inevitable. Although a weapon factory doesn't test the bullet that's sold to you, it does test copies of them in isolation. Behaviour is pretty much predictable. Every now and then one will fail. I didn't serve in the army, but I heard sometimes that a gun might blow in your face when the bullet's stuck. Nothing different her.
Regards, Carel-Jan
===
If you think education is expensive, try ignorance. (Derek Bok)
===
At 09:05 AM 3/20/2004, you wrote:
>You're the man...
>
>Agreed - that one is easy. But what about real test scenarios? The last
>sentence in Dave Ensor's Oracle Design book from 1997 says:
>
>"It really is absurdly difficult to design a piece of code that will
>completely meet a totally unknown requirement."
>
>Funny as it is, this is increasingly what we're asked to do. Maybe not
>"totally unknown", but trying to translate business requirements into
>scientific technical stuff is not easy, to say the least. Impossible? That
>depends on your definition of impossible, of course.
>
>But think about implementing Oracle Apps (or SAP or Siebel or JDEdwards or
>Axapta or whatever). 42 modules have been bought by the customer. Some of
>them will be accessed by a known number of users in an unknown way. Some
>of them by an unknown number of users in a known way. And so on.
>
>I'm not saying it's impossible to test realistically for performance on
>such a system. I have just never seen it done. I have seen many tests,
>though. But they always end up testing a well-defined workload - and how
>many systems have well-defined workloads? And for what usage intervals?
>End-of-month? A typical Tuesday? The possibilities are many :-).
>
>Which explains why some of the World's biggest (and medium-sized) systems
>still get into trouble. Not because they didn't have the right people
>allocated to size and test the system. Not for lack of test harnesses or
>licenses to LoadRunner and other excellent products. But because you can't
>test complex dependencies and load combinations.
>
>So many systems end up being over-sized. In case of doubt, we'd better buy
>the biggest machine we can get our hands on. Let's also throw in 128 GB of
>RAM just to be on the safe side. You never know.
>
>I'm not arguing against testing for performance, availability or
>functionality. I'm just arguing that many times when eg supporters (was
>one myself for 10 years in Oracle, remember :-)) say: "Ah, but this
>situation could have been avoided if you had tested this more carefully"
>they are of course right. Water runs downhill. But could they have done it
>better themselves? In most cases I think not.
>
>OK, then you put the system into production after careful testing. It runs
>OK for a while. Then you need to upgrade from 9.2.0.3 to 9.2.0.5. So the
>heuristics (constants) of the optimizer changes a little bit, because
>somebody in Development has fixed a bug or enhanced some functionality.
>Then you should test again, right? I mean, you shouldn't upgrade unless
>you have tested...
>
>Then you hit some bug, and the fix will be put into 10g release 2. Until
>then you're advised to set _many_complex_joins_forever=42 and unset
>timed_statistics and change to AUM to work around the problem. Well, now
>you need to test again. You can't just put these changes into init.ora
>without testing it...
>
>Imagine the amount of time it could take to fully test the environment in
>these cases. It's not feasible. And what about emergency situations
>(ora-600's, security patches, etc.)? Do you wait a week or two before
>applying patches or do you just do it?
>
>And any patch is a potential de-stabilizer of your system.
>
>My stipulation is this: Sooner or later you're forced to just do it out
>there in the real world. Remember how we used to be extremely careful
>about applying patches to our Windows laptops? Now it says: Click here and
>you will have all sorts of changes done to your system. So you click,
>because you don't even want to think about going through it, test it,
>backup the system before you click, etc.
>
>It's what Oracle is doing in 10g, by the way: "You should apply this
>monster patch set. OK, done. Oracle cannot possibly know how it will
>impact your system, and you're not going to test it before clicking.
>
>Writing this, and thinking about the projects we're getting into, and
>seeing what customers are asking, I think we will increasingly end up with
>exactly the situation Dave's sentence describes.
>
>That's not because the people that need the systems are more stupid or
>more evil than their equivalents 10 years ago. It's because the complexity
>is increasing so fast (exponentially) that it has become impossible to do
>the isolation thing anymore, except in some absurdly simplistic systems
>for very, very specific and repeated usage.
>
>Yes, a dynamic system with "on-demand" resources ad libitum (yeah! and
>without new licensing models this will ROCK for software and hardware
>vendors) could address some performance needs, thus pretty much removing
>the need for that part of testing (in the perfect world of infinite
>resources). But still there's the interdependency, the serialisation, the
>functional, and other testing requirements that won't be adressed by
>infinite hardware resources.
>
>So is the grid, RAC and all that really about admitting that you cannot
>test anymore? :-)
>
>Mogens
>
>Pete Sharman wrote:
>
>>Ooohh I like it when you're rough! :)
>>Actually, given what John responded in a separate email as an example of
>>why there were complaints about lack of testing, I totally agree with
>>you. It's impossible to test for things that are out of scope. In this
>>case, testing to include a script that a user wrote him/herself that you
>>know nothing about just ain't viable. What are needed here are scope
>>boundaries. Like any good project, the testing should define up front
>>what's in scope and what's out of scope. User written scripts that are
>>in the user's login directory and not under change control for the app
>>are clearly out of scope. And if you need ad hoc query capability, then
>>maybe you need the ability to dynamically grow and shrink the computing
>>resources available to you. Heck, maybe you need a grid! :)
>> Pete
>>
>>"Controlling developers is like herding cats."
>>Kevin Loney, Oracle DBA Handbook
>>
>>"Oh no, it's not. It's much harder than that!"
>>Bruce Pihlamae, long-term Oracle DBA
>>-----Original Message-----
>>From: oracle-l-bounce_at_freelists.org
>>[mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Mogens Nørgaard
>>Sent: Saturday, 20 March 2004 9:34 AM
>>To: oracle-l_at_freelists.org
>>Subject: Re: Hardware / OS recommendation
>>And I think there never, ever can be enough testing. If anything goes
>>wrong, or if anything behaves worse than what we want or expect, we can
>>always - always - say: "Ah, they should have tested it (more)". But this
>>is NOT the case, in my opinion. It's just an easy way out for all of us.
>>A way to blame someone else, when we don't know what to do ourselves anyway.
>>[This is not an easy shot at you, Pete. But I've been wondering about
>>tests for a while. I think they're not worth much, to put in mildly :)].
>>If a test should be of any value, it should prove something. But can it
>>ever prove that your environment - your combination of online ad-hoc,
>>planned batch and ad-hoc batch - can run on a given combination of
>>thingies? No, it can't.
>>You can test and measure and judge and guess that your system can
>>sustain the IO workload the system can handle. You can pray that
>>serialisation (latches, locks, enqueues, etc.) won't get in the way. But
>>can you control batch in a Unix/Windows world? No, you can't. Can you
>>direct certain services/stuff to dedicated CPU's? Yes, but with great,
>>great difficulty.
>>In the words of my old friend Ole (sorry, that's his name. So Ole' Ole
>>sounds pretty cool...): "Benchmarks are always in-conclusive."
>>They are. They might serve the purpose of making the bosses happy and
>>feeling good in their stomach. But they will never be able to mimick the
>>real load on the system.
>>In the managed mainframe world they can usually predict fairly precisely
>>what will happen to application A if X happens and what will happen to
>>app B if Y happens.
>>No way to do that in our world. Or to be more provocative: If there
>>really was a systematic way of doing this, I would have thought it would
>>have been standardized a long time ago.
>>So lean back, Pete, and tell me what you would test before putting a
>>mixed online/batch environment into production? How the Hell are you
>>going to emulate an ad-hoc environment without "just" doing the "Yes!
>>We've done this, we've done that" routine in the benchmark?
>>I'm a bit rough on you right now, and that's not what I meant. You're a
>>rather cool man who knows his stuff.
>>Mogens
>>Pete Sharman wrote:
>>
>>>Well, he did say "have to field user complaints for weeks after each
>>>move, despite testing." That immediately implies there hasn't been
>>>sufficient testing to me. :)
>>>
>>>
>>>Pete
>>>
>>>"Controlling developers is like herding cats."
>>>Kevin Loney, Oracle DBA Handbook
>>>
>>>"Oh no, it's not. It's much harder than that!"
>>>Bruce Pihlamae, long-term Oracle DBA
>>>
>>>-----Original Message-----
>>>From: oracle-l-bounce_at_freelists.org
>>>[mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Mogens Nørgaard
>>>Sent: Saturday, 20 March 2004 6:01 AM
>>>To: oracle-l_at_freelists.org
>>>Subject: Re: Hardware / OS recommendation
>>>
>>>Of what?
>>>
>>>Pete Sharman wrote:
>>>
>>>
>>>
>>>>More testing? :)
>>>>
>>>>
>>>>Pete
>>>>
>>>>"Controlling developers is like herding cats."
>>>>Kevin Loney, Oracle DBA Handbook
>>>>
>>>>"Oh no, it's not. It's much harder than that!"
>>>>Bruce Pihlamae, long-term Oracle DBA
>>>>
>>>>-----Original Message-----
>>>>From: oracle-l-bounce_at_freelists.org
>>>>[mailto:oracle-l-bounce_at_freelists.org] On Behalf Of John Flack
>>>>Sent: Saturday, 20 March 2004 12:54 AM
>>>>To: oracle-l_at_freelists.org
>>>>Subject: Hardware / OS recommendation
>>>>
>>>>We are currently running 8.1.7 databases on a 4 year old Dell Pentium
>>>>machine, under SCO UnixWare. This is the last supported version of
>>>>Oracle under UnixWare, and since the hardware is getting old in internet
>>>>years, we're thinking of getting new hardware running a supported OS for
>>>>Oracle 9i R2 or 10g. I'm the official DBA, but my system administrator
>>>>has been wearing an Asst. DBA hat doing much of the day to day work.
>>>>
>>>>The SA wants to get a low-end Sun SPARC machine running Solaris, since
>>>>the price of these has come down to around the same price as the sort of
>>>>high end Intel or AMD machine that we would normally use as a server. I
>>>>would normally vote for the Intel/AMD solution running Red Hat or SUSE
>>>>Linux, since we already run several of those. And maybe there are some
>>>>low-end machines from HP or IBM (or someone else) that we should
>>>>consider.
>>>>
>>>>One thing I'd definitely like is an OS that Oracle will support for a
>>>>long time. We started on old SCO Unix, moved to SCO Openserver when
>>>>Oracle stopped supporting it, moved to UnixWare when Oracle stopped
>>>>supporting Openserver, and now have to move again. Oracle is Oracle,
>>>>and we've never had much of a problem with the database stuff - an
>>>>export and an import, and we've been good to go. But the shell scripts,
>>>>COBOL and C programs have required tweaking every time we moved.
>>>>Nothing major, but just enough to have to field user complaints for
>>>>weeks after each move, despite testing.
>>>>
>>>>Suggestions, anyone?
>>>>----------------------------------------------------------------
>>>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>>>----------------------------------------------------------------
>>>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>>>put 'unsubscribe' in the subject line.
>>>>--
>>>>Archives are at http://www.freelists.org/archives/oracle-l/
>>>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>>>-----------------------------------------------------------------
>>>>
>>>>----------------------------------------------------------------
>>>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>>>----------------------------------------------------------------
>>>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>>>put 'unsubscribe' in the subject line.
>>>>--
>>>>Archives are at http://www.freelists.org/archives/oracle-l/
>>>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>>>-----------------------------------------------------------------
>>>
>>>
>>>----------------------------------------------------------------
>>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>>----------------------------------------------------------------
>>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>>put 'unsubscribe' in the subject line.
>>>--
>>>Archives are at http://www.freelists.org/archives/oracle-l/
>>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>>-----------------------------------------------------------------
>>>
>>>----------------------------------------------------------------
>>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>>----------------------------------------------------------------
>>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>>put 'unsubscribe' in the subject line.
>>>--
>>>Archives are at http://www.freelists.org/archives/oracle-l/
>>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>>-----------------------------------------------------------------
>>
>>----------------------------------------------------------------
>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>----------------------------------------------------------------
>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>put 'unsubscribe' in the subject line.
>>--
>>Archives are at http://www.freelists.org/archives/oracle-l/
>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>-----------------------------------------------------------------
>>----------------------------------------------------------------
>>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>>----------------------------------------------------------------
>>To unsubscribe send email to: oracle-l-request_at_freelists.org
>>put 'unsubscribe' in the subject line.
>>--
>>Archives are at http://www.freelists.org/archives/oracle-l/
>>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>>-----------------------------------------------------------------
>
>----------------------------------------------------------------
>Please see the official ORACLE-L FAQ: http://www.orafaq.com
>----------------------------------------------------------------
>To unsubscribe send email to: oracle-l-request_at_freelists.org
>put 'unsubscribe' in the subject line.
>--
>Archives are at http://www.freelists.org/archives/oracle-l/
>FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html
>-----------------------------------------------------------------
Regards, Carel-Jan
===
If you think education is expensive, try ignorance. (Derek Bok)
===
-- Archives are at http://www.freelists.org/archives/oracle-l/ FAQ is at http://www.freelists.org/help/fom-serve/cache/1.html -----------------------------------------------------------------Received on Sat Mar 20 2004 - 10:15:24 CST
 |
 |