Andrew Clarke

A funny from a system test server today, which is being refreshed. The database is up and the oracle account can login through SQL*Plus without a hitch. However, when we attempt to connect through the OPS$ accounts, via sudo, we get the following error:

ERROR:
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
SVR4 Error: 2: No such file or directory

The first thought which occurs is the ORACLE_SID is wrong but it is not. Although the answer does lie in the .profile as it is the ORACLE_HOME which is wrong. The oracle account has an ORACLE_HOME of /u01/app/oracle/products/9.2.0 and the other accounts have /u01/app/oracle/product/9.2.0. Subtle, eh?

I think the difference occurred because Oracle was originally installed on this server at an earlier version and then subsequently upgraded, whereas the .profile files were copied from a server which had had a greenfield installation of Oracle 9iR2. The default path suggested by DBCA is definitely app/oracle/product/n.n.n and has been for quite a while. I don't know where the /products/ variant originated; judging by the relative number of Google hits, /product/ is the industry standard. Perhaps it didn't always used to be.

I'm now working in a client site. One of the differences is that each development desktop bears a smug "Ubuntu - Linux for human beings" sticker. This is the first time I have really tangled with Linux. I'm not going to risk the wrath of Verity Stob by detailing my journey into the heart of the penguin. But I thought it would be worthwhile documenting my experiences with installing Oracle 10gR2 on Ubuntu 8.04 (AKA Hardy Heron).

Ubuntu is touted as a user-friendly flavour of Linux, and certainly the graphical desktop is welcoming to people used to Windows. However things get pretty gnarly pretty quickly as soon as you want to do anything off piste. And Oracle is not supported on Ubuntu ( the supported distros are Suse, RHEL, Asialux and Oracle's own Unbreakable Linux) things go very off piste indeed. In the absence of official documentation we're thrown on the resources of the internet. There is lots of information out there - the sort of people who love Linux are the sort of people who love the web - but it is often written with a presumption of familiarity in Linux. In amongst the shedloads of helpful advice there are opaque sentences such as this): "to install lesstif2 you will need to use 'Adept' to enable the universe repository or edit /etc/apt/source.list" (unfairly quoting Todd Trichler out of context). I know I should have devoted myself to studying the Linux architecture before I started but I really needed to install Oracle now.

The first guide Google turned up was Luca Mearelli's Installazione di Oracle 10g su Ubuntu Linux which as you might have guessed is written in Italian. I found this quite distracting as anything in that language sounds like opera.Madame Butterfly. Act 2. Scene 1.Whilst she awaits the return of Lt Pinkerton Koko-chan amuses herself by installing Oracle on different Linux distros. She sings the aria Impostazione dei parametri del kernel.It was the setting of kernel parameters which drew me up short. There was no explanation (and even if there had been, my Italian would not have been up to translating it). Should I be changing kernel values on the say-so of a random Google hit?

So I surfed a bit more and found Installing 11g on Ubuntu Hardy Heron by Pythian's Augusto Bott. This article is so good that it has been ripped off by plagiarising sites. Augusto's guide included the same changes to the same kernel parameters as Luca's guide but it included explanations (through a link to his earlier article about installing Oracle 11g on Ubuntu Feisty Fawn). Although he wrote his guide for 11g it works just as well for 10g. There were still a few things which caused me some puzzlement and I will discuss those below.

I wasted a lot of time trying to install 64-bit Oracle (because we have 64-bit desktops). I could download it and run the Installer but I couldn't get OUI to link the database packages. The problem is:

  INFO: /usr/bin/ld: skipping incompatible 
/u01/app/oracle/product/10.2.0/db_2/lib/libsql10.a
 when searching for -lsql10

After a couple of hours fruitlessly downloading further packages I gave up and tried 32-bit, which worked first time without a hitch.

Annotations for Augusto Bott's guide
Step 3 applied because I was installing on the Ubuntu desktop. The gotcha is in the innocuous statement "You will have to restart your Xserver for this change to take effect." The Ubuntu desktop environment is X so the simplest way of doing this is to logout and login again. However I only discovered this fact after I issued the following command in a terminal window:

sudo /etc/init.d/gdm stop

Theoretically this should have just dumped me out to a text-mode command prompt. It didn't quite work that way so I had to resort to a hard reboot. Incidentally, another way to get out of the graphical environment is ALT+CTRL+F1 while ALT+CTRL+F7 gets you back again.

In Step 8 the wise man will take a backup of these files just in case. The suggested values seem to be common across all the install guides I read, so I trusted them :) Given that I was only installing locally I didn't bother setting the network parameters.

Step 9 is the actual install of the software. Oracle's Universal Installer is a Java applet, which is why we need to change the X Windows settings. If you have trouble with this step first make sure you have done Step 3 properly. However I still got the OUI-10025 message. More Googling threw up this piece of voodoo, which solved the problem:

clarkea@clarkea-desktop:~$ export DISPLAY=:0.0
clarkea@clarkea-desktop:~$ sudo su - oracle
Your account has expired; please contact your system administrator
su: User account has expired
(Ignored)
oracle@clarkea-desktop:~$ export DISPLAY=:0.0
oracle@clarkea-desktop:~$ xhost +
access control disabled, clients can connect from any host
oracle@clarkea-desktop:~$

If xhost or xclock works then you can run OUI.

The OUI wizard is slightly different in 10g. In particular it doesn't prompt for ORACLE_BASE and it defaults the paths to hang off your $HOME directory. You may want to change the location to /u01/app.

For me the actual install and linking process did not take nearly as long as Augusto suggests it will. Obviously our machines are a lot more powerful than his :)

Finally, if you apply the scripts Augusto suggests you will need to change the paths to point to 10.2.0 instead of 11.1.0.

To apply the 10.2.0.4 patch you just need to repeat the process.
Further References
Augusto's earlier article on installing on Ubuntu Feisty Fawn (which has some additional explanation of the parameter tweaking)

Oracle Release for 10gR2 on Linux note

Oracle Install Guide for RHEL4 and SLES9

Today, 4th July, is Independence Day. I know this because Tech Republic has sent me an e-mail of special Independence Day offers. Only not that special, as the list seems to be the same list of offers they mailed for Father's Day. At least that made sense: after all, nothing says "You're the best dad in the world" quite like a gift of the Administrator's Guide to TCP/IP But what sort of patriot celebrates Independence Day by settling down with IT Professional's Guide to Policies and Procedures, Third Ed instead of fireworks, corn dogs and "light tasting" beer? Probably the sort of patriot who reads Log Buffer, so I'd better get on with it.

Staying with the Independence Day theme Curt Monash picks up on a humourous press release from data warehouse appliance vendor Dataupia. It's in the form of a Declaration of Data Independence and is probably funnier if you're American.

In the UK there have been rumours that the government is planning a giant database to track all our telephone and internet activity. On the BCS blog David Evans skips the ethical dimensions and looks at some of the practical considerations. However, the most pertinent point is made by Matthew in the comments: "How many days after the launch of the Big Brother Database ... do you think it will be before someone loses a disk or backup tape full of its contents?"

I'm just an Oracle person, which according to Max Kanat-Alexander means I suffer from Oracle-itis. Apparently symptoms include not being able to recognise the difference between NULL and an empty string, and thinking that one thousand items is a sensible limit for an IN clause. Kevin Closson posted a suitably withering response in his series on things which doth crabby make.

Anyway, doing the Log Buffer has given me - with the assistance of David Edwards and Google - with some exposure to other databases and other ways of doing things. For instance, Leo Hsu and Regina Obe wrote about inheriting tables in PostgreSQL. This is quite a neat idea."lets say you developed a timesheet app for an organization and each department insisted on having their own version of the app and each along with the basic fields needed to track some additional ones of their own. Then higher forces came in and said I need to know what everyone is doing, but I don't need to see all that other crap they keep track of.. Two options come to mind - create a bunch of views that union stuff together or institute a round-up-the-children-and-adopt-them program."In Oracle the only option would be the view (possibly of the materialized kind). Off the top of my head I can't recall a case where I could have used this but it's definitely the sort of capability it's nice to have in your back pocket.

Another intriguing idea which has no parallel in Oracle is the MySQL Sandbox. This is a framework for testing features of different versions of MySQL without jeopardising our primary environment. Its developer, Giuseppe Maxia, The Data Charmer announces that MySQL Sandbox 2.0 has been released.

Regardless of which database you use performance is always an issue. Hubert Lubacewski has a offers a technique for identifying who is is trashing the performance of your PostgreSQL database. Arjen Lentz posts a MySQL script for finding useless indexes. The problems are the same, but the metrics are very different from the ones I'm used to in Oracle: "The query returns all indexes in a db where the cardinality is lower than 30% of the rows, thus making it unlikely that the server will ever use that index." Peter Zaitsev on the MySQL Performance Blog discusses the importance of identifying where the bottlenecks are. There's no point in a web developer tweaking CSS or JavaScript if the real problem lies in the database access layer: "get real numbers for your application before you decide." Ken Downs , the Database Programmer, has some general SQL advice on designing your web application's data model. Mr Oracle Index himself, Richard Foote, gives us his 3 Steps To Performance Tuning.

Transaction management is one of those things which varies considerably from product to product. Many Oracle practioners still think MySQL doesn't have transaction management. This is a canard Pythian's own Keith Murphy lays to rest by writing on transactions in InnoDB. In a related post covering transaction basics says he may write further pieces on "the major storage engines and their transactional characteristics". I presume he means the different MySQL storage engines but I think there's scope for a series which covers all the different database products.

For instance, nested transactions in SQL Server strikes me as asking for trouble. Which is why Kalen Delaney rails against the loss of the Sysprocesses.open_tran column in the SQL Server 2005 metadata. "Sysprocesses contains a columns called open_tran which reflects the transaction nesting of each session. If a session issues four BEGIN TRAN commands, with no COMMITs or ROLLBACKs, their session will have an open_tran value in sysprocesses of 4. Any open_tran value greater than 0 might mean that a transaction is holding locks and blocking other processes, or it might be keeping the transaction log from being cleared. If you ever notice open_tran values in higher than 2 or 3, it's a pretty good indication that a developer doesn't know much about SQL Server transaction management."Back to Pythian where Sheeri Cabra reviews MONyog, a GUI monitoring tool for MySQL. Overall she is favourably impressed: "MONyog is the best out-of-the-box GUI monitoring tool for MySQL that I have seen.” Although she does have reservations about its logging. Personally I think the name is a mistake: it sounds too much like something out of H P Lovecraft.

Some Oracle stuff now. Chen Shapira, the not-so-simple DBA puts her Statistics degree to good use by building a custom aggregation function that will return a random salary using Oracle's Data Cartridge extensibility features."The main challenge was to make the aggregation truly random....Suppose I have three rows. The way aggregation works, I first take two rows and flip a coin to pick one. Now I have a current value - and I have to take the third row and decide if I want to keep the current value or the new one. I can’t flip the coin again - because if the third row has 50% chance to be selected, this means the first and second rows only have 25% chance each. Not fair. So I need to give the third row 1/3 chance, and the current value 2/3."On Oracle Base, Tim Hall demontrates the long-overdue support for case-sensitive passwords which Oracle have introduced in 11g.

Oracle has acquired the IKAN tool CWD4ALL and they're going to use it to give SQL Developer a decent modelling support capability. I would have though there would be more excitement about this in the blogosphere (certainly the ODTUG Designer listerserver has been cock-a-hoop) but only Dietmar Aust seems to have picked it up. Perhaps hardly anybody cares about modelling, in which case TOAD's marketshare is safe.

Last week I was judging abstracts for the UKOUG 2008 Conference, and there were three submissions for sessions on best practices in programming with ApEx. Alex Gorbachev (Pythian, again) shows why these talks are necessary with an example of poor SQL taken from the official Oracle documentation.

Lot's of people are asking questions. SQLDenis asks rhetorically Sybase IQ Is A Columnar Database, Why Should I Care?"What does this mean? This mean that the data is stored in columns and not in rows. Inserts are slower that a traditional row based database but selects are many times faster (up to 50 times). The good thing about this technology is that the SQL looks the same, the only difference is that the data is stored in a different way." Robert Hodges at The Scale-Out Blog wants to know, what's your favorite database replication feature? Call me shallow, but it's not a topic to which I'm given much thought. I can tell you my five all-time top favourite cover versions instead.

Meanwhile Jon Emmons poses the question Ever wonder what your DBAs really do?. It turns out there's more to the job than drinking coffee, swearing at developers and losing the backup tapes. Who knew?

Of course, DBAs have plenty of reasons to swear at developers. In Extreme Makeover - Database Edition CrazyDBA shows us his scars from a SQL Server version upgrade:"Saturday morning, migrating from "old prod" to "new prod". We finish up during the afternoon. On Sunday evening (yes, more than 24 hours later), we are notified that the system is not performing properly. We double check things on our end and everything seems to be working, well, except that the duration for some queries have gone from three seconds on "old prod" to twelve minutes on "new prod". Ouch. Our team investigates a bit further and escalates the issue to the (sleeping) onsite team, who pick up their research on Monday morning.

What do we do first on Monday morning? Well, we go to the new test system and run the query. It takes eight minutes. Turns out development is slow as well. Surely someone noticed this during testing, right?"Er, wrong.

Rick Heiges asks It's Q3 - where is SQL Server 2008? To make him happy (and Mr CrazyDBA even happier), according to Jason Massie there's a rumour that SQL Server 2008 is due to ship next week (or this week if you're reading after the weekend).

From the new releases to some ancient history. Willie Favero comments on an article about DB2's 25th birthday from Information Week. It's interesting to see what counted as a new feature in those days: "You could dynamically add tables or change tables without taking the system down. It doesn't take much imagination now to see this was a huge leap forward," recalled Don Haderle, chief architect of DB2.

Back to the future. Over at the IT Toolbox Lewis Cunningham has his head in the clouds. Or rather Cloud. This is a neat summation of all the main players in Cloud databases. Cloud computing is a rather attractive idea, but I think there is some way to go before it is a practical solution for business. Web access is still far from pervasive or guaranteed, and as the Register pointed out this week, there are still some kinks in the business model to iron out. Lewis describes Amazon as the 800lb gorilla in the cloud space (stratosphere?) and the Register also has a good piece explaining Amazon's interest in the technology.

In his Data Migration blog Johny Morris (no, not that one) invites us to consider the benefits of meetings, in this case Data Quality Rules meetings: "Use them not just instrumentally to solve the issues in front of you but also to build the team that jointly will have uncover all the knowledge hidden in the organisation." Exactly the sort of benefit which will be hard to realise when we are all working in the Cloud and never visit the office.

Finally, nothing to do with databases but I'm sure relevant to us all (at least those who are still office bound), Suzanne Thornberry at Tech Republic writes about the health risks IT professionals run. These include such things as eye strain, bobblehead syndrome and seated immobility thromboembolism (SIT), which is like DVT only worse. So stop reading this and go do something more healthy instead!
Postscript
The Log Buffer is a community activity facilitated by Pythian. Find out more.

According to The Register the next iteration of Sun's Niagara chip will have 16-cores and 16 threads per core . Apart from the mind-boggling number of threads which will become available in an eight socket 1U rack, the licensing implications are a bit of a facer for Oracle. Soon a server with a single chip in it could incur a sixteen CPU license. At least at the moment if customers don't want to pay Oracle's multi-core fees they have the option to tear out some chips. But that's not an option with Niagara 3. Can Oracle seriously maintain a policy of selling licenses in bundles of sixteen?

Brian Tkatch has launched a thread on the PL/SQL forum about enhancements to SQL which would just basically save some typing: Things i wish SQL supported. The lazy man's list. This is quite a revealing thread, because it is always interesting to see what shortcuts people would like to take. It's a bit like peeking inside the medicine cabinet in other people's bathrooms (not that I would ever do that).

My personal wish is for:

select * {-empno} from emp;

That is, select all columns from the EMP table except EMPNO. This would be particularly useful for querying tables with BLOB columns in SQL*Plus.

As the thread as grown it has turned into a discussion of SQL theory ("conceptually, (using Venn diagrams) the tables/views are the circles, and the predicates define in what way the circles overlap") which requires too much concentration. The thread was supposed to be about laziness!

The patron saint of programmer laziness is Larry Wall, the inventor of Perl:"The virtues extolled for Perl programmers are laziness, impatience, and hubris. Together, these admirable characteristics have led to the creation and use of many publicly accessible Perl modules. Because of laziness, programmers would rather write modules than repeat a procedure over and over (and would rather use modules written by other people than write new code from scratch). Because of impatience, programmers write consolidated code that is flexible enough to anticipate their future needs. And because of hubris, programmers share their triumphs with the rest of the Perl community and continually tweak their modules until they're the best they can be."

The problem with proactive laziness is that it can be hard to estimate how much effort will be saved later by putting in some extra effort now. Plus, writing automating utilities and code generators can just be a seductive form of procrastination. It feels like work but we aren't moving forwards. In the end we spend so much time sharpening the axe that we never get around to cutting down the tree. So the trick is to only automate the things we know it will be worth automating. This means doing something the plain way at first. Only when we get to the second or third cut'n'paste should we consider whether we need a parameterised module instead. The important thing is to automate early, in order to derive the maximum return on the work.

I am currently practicing cut'n'paste programming in a test data generator. I could refactor my code to drive off an array but re-editing my package to populate a collection will be a PITA. I should have done it some time ago, but I failed to realise just how many additional datasets I was going to need. At this point the ROI on the automation is quite small. So I have chosen to continue paying the find/copy/edit tax rather than spending half a day to figure out a better way of doing things. In the long run I will have expended more effort but in the meantime I keep making progress towards the main goal.
Update
Over on the Artima site Jeremy Meyer has written an article on Why it is better to be lazy.