Feed aggregator

SQL Gone Wild!

Eric S. Emrick - Mon, 2006-09-04 14:22
Ever see something so inefficient it evokes images of grape stomping to produce wine? I have, in Oracle 10g no less. A colleague of mine brought me a situation the other day that made me do a double-take, no triple-take, on a 10046 trace. The scenario involved a single row delete from a table using the index associated with the primary key on said table to delete the row, simple right? Well, the delete hung. The 10046 showed "db file sequential reads" spewing at a very rapid clip. The process was reading a child table that contained a column that referenced the primary key of the table being deleted. Okay, this is to be expected. We don't want to break our self-imposed business rules by orphaning child records. So what is my beef with this situation?

The child table had millions of rows that would have been orphaned had the delete succeeded. Keep in mind the constraint was NOT defined with ON DELETE CASCADE. Also, a single column index on the child table was associated with the child key. The stage was set for a swift and proper decline by Oracle to perform our delete. But this did not happen. Oracle was visiting ALL of the child rows then returning ORA-00292 "... - child record found." Yes, each and every child index entry was being visited. My colleague opened as SR with a very elegant little test case that reproduces the problem. Here it is. Try it for yourself and watch the trace with wonder and amazement. We have performed the test in 8i, 9i and 10g with the same results.



-- Insert approximately 1 million records into CHILD
for i in 1..20 loop
insert into child select * from child;
end loop;

alter session set events '10046 trace name context forever, level 12';


Why doesn't Oracle stop once it encounters the first index entry indicating a foreign key violation has just occurred? Isn't a single found entry sufficient to fail my statement? It seems a bit indulgent to check each and every child row irrespective of my barbaric attempt to break my own business rules. Is it a classic case of stupid is as stupid does? Nope. It is a good old fashioned Oracle bug.

By the way, the Oracle support analyst recommended putting the index associated with a child key in a read only tablespace as a workaround. Think about that for a second...

Random training related thought

Stephen Booth - Mon, 2006-08-28 15:50
A thought just crossed my mind. I wonder if there is a market for short (one day) task oriented training for the sort of things that aren't normally covered in general courses.For example I'm aware that there are training courses that mention standby databases and certainly there are courses that cover RMAN, at least for basic backup and recovery. I'm not aware of any that actually take you Stephen Boothhttps://plus.google.com/107526053475064059763noreply@blogger.com0

Oracle Develop

Denis Goddard - Wed, 2006-08-23 21:19
I've offically been a PHB for a long time -- long enough to "ossify" (as one of my old friends back at Cummins Electronics used to put it). I know this because I find myself realizing that it's been so long since I really developed anything in detail, I don't even remember what the new features are in Oracle 10g, let alone 10gR1 or R2... (though of course I'm really looking forward to 11gR2, but that discussion will have to wait!)

As it turns out, in the next week or two, I should be able to turn my attention back -- finally -- to ADE 3.2.3, where I'm doing some rather detailed work on PL/SQL and in the Perl/XS layer. The kind of thing where you have to spend a good hour just loading your brain's cache with the details... get into the zone... and stay there for a solid 10-hour block. What Tom Christiansen called Interface Zen
Ah yes, by god, I do so miss being a programmer!

Anyway, I'm seriously considering attending Oracle Develop 2006. First, Oracle keeps coming up with tools and technologies and I have to say, in my opinion we do a good job of being driven by customer demand. Plus, the Moscone Center expos are one of the few things I really miss about living on the SF Bay peninsula.

This could be seriously fun!

OpenWorld "Schedule Builder" - Get First Dibs On Your Favorite Sessions

Christian Shay - Wed, 2006-08-23 20:17
Shay Shmeltzer's recent blog post reminds me to alert you that things are first come first serve at OpenWorld this year! Use the new Schedule Builder tool to get first dibs on your favorite .NET Oracle Develop sessions!

Anyway, Shay said it so well, and has such a great first name, that I will just plagiarize from him shaymelessly:

"But the conference is two months out, why should I do it now?" - you might be asking.

Well the reason is that this is the only way you can guarantee you get to see the sessions you want to see. This year people who register for sessions ahead of time will get into the room before people who just show up. Considering the fact that this is going to be the biggest OOW ever in terms of number of attendees - you want to secure your place. Also, I know how it goes when you don't schedule, you start the conference spending the first hour of each day trying to figure out which session to go to - and by the time you end your daily planning you already missed the morning session. Scheduling your sessions now will also help us better plan rooms allocation for popular sessions.

So do us all a favor and start scheduling now.

Thank you, Shay!

Personalization and the MDS Database Repository

Andries Hanekom - Wed, 2006-08-23 04:04
All declarative User Interface components are stored either in XML files, in a format defined by MDS (Meta Data Services) Schemas, or in the MDS repository tables. When a personalization is created through the OA Personalization Framework it is added on top of the base product meta data. The personalization does not overwrite the existing base product UI and are therefore preserved during upgrades and patches. The MDS repository is supported by the JDR_UTILS PL/SQL package, used to query and maintain the repository:

SQL> set serveroutput on

PL/SQL procedure successfully completed.

The MDS database repository consists out of four tables:
  • JDR_PATHS: Stores documents, packages and there parent child relationship.
  • JDR_COMPONENTS – Stores document components.
  • JDR_ATTRIBUTES – Stores attributes of document components.
  • JDR_ATTRIBUTES_TRANS – Stores translated attribute values of document components.

Training Class (Final Day)

Eric S. Emrick - Tue, 2006-08-22 20:38
To round off the material covered in this class the following topics were covered today:
  1. Tuning Block Space Usage.
  2. Tuning I/O.
  3. Tuning PGA and Temporary Space.
  4. Performance Tuning: Summary.

I found the Tuning I/O lecture somewhat interesting. The first portion of the lecture focused on the advantages and disadvantages of the various forms of RAID protection. While informative, I could've spent 5 minutes on Google had I not already been armed with the knowledge of this technology. The remainder of this lecture focused on ASM (Automatic Storage Management). This rather non-trivial feature in 10g sounds very cool; define some Data disk group(s) , the relevant protection and striping granularity and let Oracle do the all of the I/O tuning. Of course, this is a severe over simplification of what it really does (or doesn't, as your mileage may vary). But, the point is, this feature is supposed to free the DBA from the often times laborious chore of tuning the I/O subsystem. Truthfully, I think the degree to which Oracle touts the hands-off nature of this feature is overstated; especially for busy production systems. I, nor anyone in the class, had worked with the product. Consequently, I feel there are probably very few shops out there migrating their production databases to ASM. Is it more of a political battle? After all, if DBAs will be able to someday create and manage the logical volumes/file systems this might make the System Administrators feel a little encroached upon. It is just a hunch, but widespread conversions to ASM will probably not happen anytime soon. Anyone reading this blog have any good/bad experience with ASM in a production environment? I am very interested in your feedback.

The most engaging lecture of the day was the Tuning Block Space Usage. I am really keen to the Automatic Segment Space Management (ASSM) feature. This feature warrants serious consideration given the upside: free list elimination and a considerably more robust approach to reusing blocks for inserts. As much as I liked the discussion on ASSM, the subsequent topic grabbed my utmost attention: segment shrinking. What a great (and might I add way overdue) feature. If one of my production environments was on 10g today I could see using this tool to reclaim vast amounts of space in some of my very large heap tables, index-organized tables and indexes. Oracle claims that the majority of the work can be done online. Moreover, the indexes associated with your heap tables are still usable even after the row movement inherent to the SHRINK has completed. I like the idea of having the freedom to perform these "online" activities, but I still prefer to perform these kinds of operations during quite periods. The course material gives a fantastic, albeit brief, description of the mechanics. Very nice Oracle! Once again, are there any readers of this blog that have experience with this feature and want to share your experiences?

The final two lectures, Tuning PGA and Temporary Space and Performance Tuning Summary, were good, but not great. The material seemed to belabor a few points.

In summary, if you are considering taking this course I think you are best served if you do not have much 10g experience in production environments. If your experience with 10g and some of the "tuning" features is even moderate, I recommend you not take the course. Your time would be better spent reading up on this material in the Oracle documentation set.

Eric's rating of the course: B+.

Training Class (Day 3)

Eric S. Emrick - Tue, 2006-08-22 20:36
Another day of training is in the books. What was on today's menu?
  1. Tuning the Shared Pool.
  2. Tuning the Buffer Cache.
  3. Automatic Shared Memory Management.
  4. Checkpoint and Redo Tuning.

Apparently, Oracle is migrating some of its serialization protection from latches to mutexes. For example, the structures previously protected by the Library Cache Pin latch are now protected by a mutex and evidenced by the cursor:pin S wait event. Actually there are several new mutexes and mutex related wait events new to 10g. For example:

- cursor:mutex indicates mutex waits on parent cursor operations and statistic block operations.

- cursor:pin events are waits for cursor pin operations (library cache pin now protected by mutex).

There are a couple interesting facts about Oracle and mutexes. A mutex get is about 30-35 instructions, compared to 150-200 instructions for a latch get. Also, a mutex is around 16 bytes in size, compared to 112 bytes for a latch in Release 10.2 (in prior releases, it was 200 bytes).

One of the appeals of the mutex, per the documentation, is the reduced potential for false contention. That is, a mutex can protect a single structure; often times stored with the structure it protects. However, latches often protect many structures (see cache buffers chain latch) and can yield what the documentation calls false contention. It is called false contention because "the contention is for the protection mechanism rather than the target object you are attempting to access." This all sounds really great, right? Well, maybe. If Oracle goes to more widespread use of mutexes instead of latches to protect target objects that would be a boatload more mutexes. I am sure the porters at Oracle are not intending to use mutexes exclusively in the future. But, I can see where contention in Oracle could be dramatically reduced at the cost of CPU cycles and memory. What would happen if Oracle protected each buffer with a mutex? While each mutex is less expensive with regard to memory and CPU than an individual latch, you will need considerably more mutexes for each replaced latch. 50 mutexes used to replace a single latch could run the CPU up considerably for the "same" application workload.

I have one final note on mutexes. As of version a SELECT against V$SQLSTAT and searches of child cursor lists are mutex protected.

I found the Tuning the Buffer Cache discussion somewhat interesting. Unless you have been hiding under a rock the past 4-5 years, I am sure you have heard the Oracle experts preaching the notion that ratios are not very helpful in diagnosing the health of a database. In particular, the buffer cache hit ratio is frequently tagged as meaningless. A smile came to my face when I read the following excerpt from the course material:

"A badly tuned database can still have a hit ratio of 99% or better...hit ratio is only one part in determining tuning performance...hit ratio does not determine whether a database is optimally tuned..."

Oracle is finally teaching what the experts have been saying for years!

I have been to several Hotsos events/training classes. They often talk about the need to include the buffer is pinned count statistic in the tally for logical reads. These operations are simply latch-reduced logical reads. Why doesn't Oracle integrate this information into their course material or documentation set? They still only claim that db block gets and consistent gets constitute logical reads. I monitored a process recently in one of my production environments and noticed the process did 2 buffer is pinned count logical reads for every 1 (db block gets + consistent gets). That is a substantial percentage of work owed to operations not officially categorized as a measure of work by Oracle.

Lastly, the on-topic impromptu discussions were fruitful. That always makes the training session more interesting :)

SAOUG Conference 2006

Andries Hanekom - Thu, 2006-08-10 02:15

Just got some good news, I have been accepted as a speaker at the South African Oracle User Group Conference 2006. I will be presenting my paper - "Oracle Applications Framework Development - The New Frontier", which is an introduction to the world of OAF development, personalization and extension. My presentation will be a 45 minute session scheduled for 9:55 AM Wednesday 27 September, so don’t miss it.

The conference is held at Sun City and runs from the 25th to the 27th of September, if you are interested in attending visit the SAOUG website at: http://www.saoug.co.za

Search Terms

Stephen Booth - Mon, 2006-08-07 16:59
Doug Burns recently posted in his journal a list of common search terms that people have used where his journal appeared in the results. I figured I'd post mine (according to Statcounter.com). Fortunately (or unfortunately) mine are not anywhere near as exciting as his.Perc. Search Term5.80% ora-074454.35% orapwd invalid username password2.90% exception signal: 11 oracle2.90% alter user sys2.90Stephen Boothhttps://plus.google.com/107526053475064059763noreply@blogger.com0

Problem with PUTTY connection to SUSE9

Wijaya Kusumo - Fri, 2006-08-04 06:22
When I tried to connect to SUSE9 via Putty from my windows 2000, for some reasons I kept getting the Access denied message. Since connecting from Cygwin is ok, then the problem should be on the Putty. After searching for a while, here is how I fixed the connection issue: 1. Update my Putty to the latest version: 0.58. 2. Open Putty. Load the Saved Session to the SUSE server. 3. In the Category

Instructive Presentation on Logical I/O Mechanics

Eric S. Emrick - Tue, 2006-08-01 17:40
If a picture says a thousand words than a good animation can say ten thousand. Check out this offering by Julian Dyke. His presentations relating to Oracle mechanics still reign supreme in my book. Once you see a mechanical concept "in motion" you simply don't forget it. What a great didactic device. Anyway, I just wanted to pass this along. Enjoy.

Training Class (Day 2)

Eric S. Emrick - Wed, 2006-07-26 08:38
The second day of training was much better than the first. I suspected it would get better based on the material to be covered. The topic set de jour was:

  1. Metrics, Alerts and Baselines.
  2. Using Statspack.
  3. Using Automatic Workload Repository.
  4. Reactive Tuning.

Having limited exposure to 10g in any true production environment, I found 75% of these topics interesting (Statspack chapter was not valuable to me). I really like what Oracle has accomplished with 10g with regard to the gathering and reporting of statistics and metrics (the rates of changes for given statistics). About 5 years ago I wrote a utility for 9i that allowed me to compare Oracle-captured statistics and wait event durations to similar reference points. This utility, I dubbed AppSnap (written in PL/SQL), captured the statistics and wait event durations each hour and calculated and stored the deltas in a separate tablespace. This permitted me to compare what is considered "typical" load to current load and evaluate the deviations rather quickly. I wrote a Unix shell script reporting tool called Instance Health that reports each hour the deltas as they relate to what I call peer hours. For example, each hour a report is generated as a text file and stored in a log directory. The most previous delta is compared to the same hour of day for the past 30 days, the same hour of day and day of week for the past 12 weeks and against all hours for the past 30 days. This has proved to be very valuable for detecting systemic anomalies after application upgrades, etc.

Okay. Now Oracle has come along with 10g and provides the same functionality (albeit not free). I appreciate the graphical conveyance of this type of analysis provided by Enterprise Manager. Shoot, Oracle even calculates the variance within the sampled timeframe for each metric. This is really cool because you can easily write a query that can ascertain if some metric is statistically anomalous (i.e. +-3 standard deviations). At first glance, some of the AWR reports are not very intuitive. But, the more you stare at them the more sense they appear to make. The Active Session History reporting is also a very nice feature (once again, not free).

If you already have considerable work experience with AWR/ASH/ADDM then this class probably won't provide you much value. The course does go into the mechanics of the data capturing and touches rather superficially on the reporting capabilities. So there is a good chance you probably have more knowledge about these products than this class affords. However, if you are like me and have yet to dig in your heels on a 10g production environment this class could serve as a very nice primer.

Well, I am off to day 3 of this training class.

Oracle Workflow Notifications & OAF

Andries Hanekom - Wed, 2006-07-26 03:39

If you wanted to display HTML formatted content in an Oracle Workflow Notifications before the release of FWK.H you had to use one of the following options:

  • PLSQL Documents - 32K size limit
  • PLSQLCLOB Documents – CLOB

PL/SQL development of HTML notifications that contains tables and complex layouts can be difficult and very time consuming. It requires the developer to have a very good knowledge of HTML and makes the implementation of a standard “look and feel” very difficult.

With the implementation of FWK.H we are able to leverage the power of the OAF to create multifaceted Workflow notifications. Also called JRAD notifications, FWK embedded regions is nothing more then a normal OAF region displayed in the body of a Oracle Workflow notification.

Incorporating an embedded FWK region in a Workflow notification consists of the following steps:

  1. Create an OA Component Region containing the Headers, items and tables you want to display in the body of you notification. Developing and implementing this region is done in the exact same way you would when creating a region for an OAF custom page. Remember to create a dedicated Application Module (AM) for your region that will contain all the View Object (VO) required for your layout.
  2. In core applications, create a SSWA jsp function where the WEB HTML call is: OA.jsp?page=/companyabc/oracle/apps/xx/module /webui/XxRegionNameRN
  3. Create a new Workflow attribute:


XX_PO_DOCUMENT_RN_NTFN is the custom function created in core apps and &PO_HEADER_ID is a workflow attribute I am passing back as a parameter to the region's controller to initialize my VO.

4. Add the new attribute to your message:


Save your workflow and initialize a new instance, the notification body should now be populated by the OAF region. No messy HTML coding, quick, easy and you can reuse the region in your OAF custom pages.

VNI-2002 File operation error when submiting jobs through Enterprise Manager

Neil Jarvis - Tue, 2006-07-25 09:39
If you’ve been able to submit jobs through Enterprise Manager and then find that they are failing with VNI-2002 file operation error check out $OH/network/agent and see if you have inadvertently created a temp directory. I did, to store some files for backup. Either remove this directory or rename it. Then restart your agent and you should find your jobs now run

Training Class (Day 1)

Eric S. Emrick - Mon, 2006-07-24 20:29
As I mentioned in my last post, I am attending an Oracle training class this week. The class is Oracle Database 10g: Performance Tuning. Having been to the Oracle 10g New Features for Database Administrators class last year, I was hoping for a substantially more in-depth look at 10g instance tuning in this class.

The first day was just okay for me. I learned a few things but I felt the class dragged a bit: 30 minutes getting to know each other and too many unrelated tangents or related, yet gratuitous, topic embellishments. I don't mind an occasional anecdotal deviation as it relates to the topic at hand, but those that are completely off topic really slow down the course. You know when you are reading a story and everything seems to flow nicely (proper balance of dialogue and narrative), then you run into a couple pages of narrative? It really puts the brakes on the interest.

I tend to evaluate courses on:
  1. The conveyed knowledge of the instructor.
  2. The presentation of the subject matter.
  3. The quality of the course material.
  4. The value of the impromptu discourse along the way (icing on the cake stuff).

Based on the first day, I feel the instructor has a good command of the subject matter and adds value to the class based on relevant experience. I have been to several Oracle classes where the instructor did nothing more than read the slides verbatim and/or appeared to have little relevant experience. Aside from the introductions, we spent the remainder of the day on two topics: a performance tuning overview and 10g statistics/wait events. The study material that accompanies the course is rather good; I have taken the liberty to skip ahead and get into the meat of the course material. I am looking forward to the class tomorrow as I feel we will be digging our heels in a bit more (or at least I hope).

Given the sparse amount of substantive material covered on the first day, I don't have any really interesting takeaways. I'll give the first day of class a B.

Microsoft and Oracle Work Together

Edward Whalen - Mon, 2006-07-24 15:12

In all the years that I have worked with both Microsoft and Oracle I have never seen so much interoperability and cooperation. This week Oracle announced support for People Tools on SQL Server 2005. See the announcement at http://www.oracle.com/corporate/press/2006_jul/peopletools-848.html. This is just one in many examples of how the interoperability between Oracle and Microsoft products is getting better. Other examples include the ability to integrate Oracle logins with Windows Active Directory and the extensive support of Oracle products on Microsoft Windows.

Oracle was the first database software available for the Windows NT platform and was also the first database software available for the 64-bit editions of the Windows platform. Oracle supports not only their database software, including RAC, on Windows, but also People Tools, Oracle eBusiness Suite software, and the Oracle Fusion Middleware products. Oracle even supports Microsoft clustering with Oracle Fail Safe.

Recently a new type of interactivity has started occurring. Oracle Grid Control monitors the Windows OS, but now does much more. Oracle is now providing Grid Control Plug-ins for the following Microsoft products:

  • Active Directory
  • BizTalk Server
  • Commerce Server
  • IIS
  • Internet Security and Acceleration (ISA)
  • Microsoft .NET Framework
  • Microsoft SQL Server

These Plug-ins allow Oracle Grid Control to monitor and provide alerting functions, so that in the event of a failure, you will be notified via email, pager, etc. Oracle has expanded Grid Control in order to provide a full service management and monitoring utility.

On the other side of the fence, Microsoft does not explicitly create Microsoft Operations Manager (MOM) management packs for non-Microsoft products, but has opened this up to the 3rd party vendors. There are several management packs for Oracle products available today. This allows the extensibility of MOM to be used with Oracle and any OS that you happen to run Oracle on.

This week I happened to be working on my upcoming “SQL Server 2005 Administrator’s Companion” book and was going through the editorial process on replication. With SQL Server 2005 replication, you can replicate data into a SQL Server 2005 database from an Oracle publisher. This allows SQL Server reporting servers and batch servers to get their data directly from Oracle using replication. In addition, it is possible to set up an Oracle subscriber to use a SQL Server publisher, thus using Oracle for reporting and SQL Server for OLTP. It is also possible to perform distributed queries between Oracle and SQL Server databases by configuring the Oracle database as a linked server within SQL Server.

There has always been cooperation between Oracle and Microsoft for using Oracle on the Windows platform, but the management cooperation has just begun. In the next few years you will most likely shop around and decide whether you like the Microsoft tools or the Oracle tools better and then implement that tool throughout your entire enterprise.

How to create the Order Entry schema after a database has been created using scripts

Neil Jarvis - Thu, 2006-07-20 11:30
If you wish to create the oe schema at a later date after creating a database you will have to run the script ‘oe_main.sql’ in $OH/demo/schema/order_entry.

oe_main.sql asks for
1. the OE user's password. The user OE will be create by the script.
2. OE’s default and temporary tablespaces.
3. The HR user password. The HR user has to have been created. So run hr_main.sql in human_resources first. The HR user is created because there are some shared objects between HR and OE.
4. And finally, the SYS password and log_path

The script creates the OE user and grants privileges then creates the objects. Final the subschema is created.

The hr_main.sql in $OH/demo/schema/human_resources asks for
1. The HR password. The user is created here
2. The default and temporary tablespaces
3. The sys password and log_path

Run this script before oe_main.sql

Please Note: This note is applicable for 9i. File locations may vary

Distributed Computing

Stephen Booth - Tue, 2006-07-18 05:02
This morning someone passed me a link to "The Eight Fallacies of Distributed Computing". Looks pretty accurate. I'd say that many of them are doubly true of RAC.Stephen Boothhttps://plus.google.com/107526053475064059763noreply@blogger.com0

OAF and Oracle Workflow

Andries Hanekom - Tue, 2006-07-18 02:52
Most of the custom development work I have done on OAF has required me to interact with Oracle Workflow, specifically to handle approvals and interfaces. The class oracle.apps.fnd.framework.webui.OANavigation provides Java wrappers for Oracle Workflow Engine's PL/SQL APIs.

The class is actually intended for use when creating and managing OA Framework Page Flows that are defined by Oracle Workflow, but can also be used to interface with Oracle eBS Workflows.

Example Code - Launching a custom Oracle eBS Workflow from a OAF page:

import oracle.apps.fnd.framework.webui.OANavigation;
import java.math.BigDecimal;

public void launchCustomWorkFlow(OAPageContext pageContext)
String wfItemType = "CUSTOM";
String wfProcess = "CUSTOM_PROCESS";
String wfItemKey = "1001-1";

OANavigation wfClass = new OANavigation();

// Create Workflow Process
wfClass.createProcess(pageContext, wfItemType, wfProcess, wfItemKey);

// Set Number Attribute: ITEM_INTERFACE_ID
new BigDecimal(1));

// Set Text Attribute: ITEM_NAME
wfClass.setItemAttrText(pageContext, wfItemType, wfItemKey,

// Start Workflow Process
wfClass.startProcess(pageContext, wfItemType, wfProcess, wfItemKey);


Eric S. Emrick - Mon, 2006-07-17 19:19
Well, this weekend I replaced my "archaic" Generation 4 IPod with a snazzy new Generation 5 Video IPod. I chose the black facing this time for a change of pace.

I really like the IPod product - versatile, convenient and just loads of entertainment/educational potential for my ~2 hours of public transit commute each day.

With more and more news and entertainment mediums adding video Podcasts this little device has become a cool way to pass the commuting time. I enjoy reading on my commute, but watching an interesting show or funny movie can pass the time at record clips. I won't be surprised if I ask the engineer to make another loop so I can finish watching my movie before heading into work :)

I wonder when Mr. Kyte will be Podcasting some material...

I'll be in Oracle training next week attending the Oracle Database 10g: Performance Tuning (Database) class. It should be a really good course. I'll be blogging some of the interesting takeaways from the class, so stay "tuned"!


Subscribe to Oracle FAQ aggregator