Re: Oracle RAC
Date: Fri, 04 Sep 2009 17:21:50 +1000
Message-ID: <877hwfcgz5.fsf_at_lion.rapttech.com.au>
Palooka <nobody_at_nowhere.com> writes:
> On 02/09/09 21:14, joel garry wrote:
>>
>> Then there's this: http://www.washingtonpost.com/wp-dyn/content/article/2009/08/27/AR2009082701518_pf.html
>>
> Exactly. A student information system does not require 24*7 operation,
> rolling upgrades and 99.999% uptime. It just needs to work decently and
> reliably, and be back up within an hour or two in the event of a disastrous
> hardware failure. It rather sounds as though OP is trying to run before he
> can walk.
>
I'm not sure your assessment of required uptimes for a student information system is accurate. Many universities are now doing a lot of on-line work and expanding their course offerings all around the world.
For example, the current project I'm doing is at a University. 80% of their students are external or distance education students and they have students in pretty much all timezone. The student information system is used to drive many of the access/authorisation processes based on what the student is currently enrolled in etc. All applications for admission, enrolment etc is handled on-line and as the business is now servicing students in many timezones, even short downtimes are unacceptable.
One of the difficulties that
the University has been forced to focus on has been improving their ICT
infrastructure to provide 24*7 service with a very high uptime.
Interestingly, the institution has moved from Oracle running on DEC/Compaq/HP Alpha servers to running RAC on 64 bit Linux servers. This configuration has been running for about 2 years now (10gR2). To my knowledge, there has been no significant issues. There were some initial problems, which all turned out to be network issues (the institution is also going through a major network upgrade). There were also some issues with reliable connections to the SAN, but all of these seem to be resolved and none of them were Oracle specific as far as I know.
The project I've all but completed (we go live on the 15th), is a new identity registration system. It runs on an Oracle RAC cluster, uses mainly PL/SQL with a Java based web interface. We use Oracle streams for the interface between the trusted soruce systems (student information system and HR system - also running on Oracle RAC) and pump out provisioning requests in XML to our provisioning system, which manages our directories (LDAP/AD) and communicates with other back-end systems i.e. creating home directories, adding/removing from mailing lists, learning management systems, etc .
I did not run into anything that was a huge issue due to the RAC environment. There was some Oracle technology that we evaluated, like rules manager, that we rejected because it just didn't feel 'mature' enough yet. I did run into some weird issues with re-loading updated plsql packages and finding that when I queried the sources in the database, they werre still the old version. Not sure what was going on there - soon after I worked out what was going on, the problem whent away. I suspect it was related to the SAN issue.
At one stage, the development cluster was set to force everything to use a single node. This was due to the network issues - communication between nodes was unreliable. sometimes, you would do something simple, like
desc mytable
and it would take 70/80 seconds to respond. Or sometimes, you would run a simple query and it would return almost instantly and then you would run it immediately for the second time and it would take minutes!
Once the underlying network issues were resolved (turned out to be a switch with a faulty port), everything has been fine.
So, from my experiences, I wold say that on the whole, RAC works very well PROVIDED your underlying infrastructure is sound. However, it is possible that a RAC based environment may reveal problems in your underlying infrastructure that were not evident before. I would also say that it is important to have good DBAs and management that is prepared to pay for the right training. One of the reasons that the move to RAC succeeded was largely due to the fact we have two very experienced DBAs and management sent them on a considerable amount of training BEFORE implementing the new architecture. I think this is essential. All they need now is another two DBAs and we may be able to get things done in a more timely manner!
The application I worked on doesn't use dbms_pipe, though e do have one small section that uses dbms_lock. I believe some of the other applications running under RAC are using dbms_pipe and things like Oracle Workflow etc. None have had any specific problems that I know of.
The only issue that was really difficult to resolve was with the HR management system. The vendor initially signed off on us running in a rAC environment. When we upgraded to their latest version, we had all sorts of performance problems and issues with user sessions not ending etc. The vendor tried to say it was due to our RAC environment and then tried to claim their product wasn't certified to run under RAC. Luckily, we had the sign-off from them that RAC was certified!
To try and resolve the issue, we did run up the system under a non-RAC configuration and the problems were still there. We also learned that other institutions running on various different RAC and non-RAC based systems on various hardware have all had similar performance issues with the new version. Therefore, I'd suggest the second really important thing to ensure is that your application vendors have all signed-off on using a RAC based environment AND ensure you have that in writing. There seems to be a fair amount of FUD out there concerning RAC. Should an application run into issues, it will likely be RAC that is labelled as the escape goat until someone proves otherwise.
I would also suggest that Oracle licensing costs have a fair amount of 'elastic' in them. Their initial quote was really over the top. However, after some strong arm twisting and in-depth analysis of what we were already paying licenses for, we ended up with a deal that wasn't much more expensive than the previous licenses. In particular, we found that we were being licensed for some Oracle technology that wasn't even being used - removing that covered almost 80% of the additional costs for RAC. I also believe Oracle licensing costs can be affected by what type of institution you are (Universities seem to get tings a bit cheaper than the private sector) and it would seem the oracle sales staff have a bit of flexibility. We also have a 'lobbying' group that represents all (most?) of the Universities and they put a lot of work into getting discounted licenses from people like Microsoft, HP, Sun and oracle. The increased Oracle RAC licenses were considerably less than the HP Tru64 licenses and maintenance agreements. combine this with the lower cost for the Linux hardware compared to SUn?IBM/HP 'high-end' servers and things don't seem as expensive as they appear initially.
Tim
-- tcross (at) rapttech dot com dot auReceived on Fri Sep 04 2009 - 02:21:50 CDT