BI & Warehousing

DBA or Developer?

Chet Justice - Mon, 2013-08-26 16:01
I've always considered myself a developer and a LOWER(DBA). I may have recovered perhaps one database and that was just a sandbox, nothing production worthy. I've built out instances for development and testing and I've installed the software a few hundred times, at least. I've done DBA-like duties, but I just don't think of myself that way. I'm a power developer maybe? Whatevs.

I'm sure it would be nearly impossible to come up with One True Definition of The DBA ™. So I won't.

I've read that Tom Kyte does not consider himself a DBA, but I'm not sure most people know that. From Mr. Kyte himself:



At the same conference, I asked Cary Millsap the same question:



I read Cary for years and always assumed he was a DBA. I mean, have you read his papers? Have you read Optimizing Oracle Performance? Performance? That's what DBAs do (or so I used to think)!

It was only after working with him at #kscope11 on the Building Better Software track that I learned otherwise.

Perhaps I'll make this a standard interview question in the future...

Semi-related discussions:

1. Application Developers vs. Database Developers
2. Application Developers vs. Database Developers: Part II
Categories: BI & Warehousing

Conditional Formatting of Calculated Items in OBIEE 11g

Chet Justice - Tue, 2013-08-13 15:24
By Victor Fagundo

Calculated items in OBIEE Pivot tables can be very useful in certain reporting circumstances, either for ease of development, or to meet specific report requirements. While calculated items in OBIEE are easy, and flexible, they do have one important drawback: they take on the data and display formatting of the fact column they are calculated against.

The most common case is the calculation of a % change across a time dimension in financial reporting ( Year over Year, Quarter over Quarter, etc.). 1   This type of calculation usually takes the form of a percent change calculation similar to below:

 (( $2 - $1 ) / $1) *100 

By default, if you perform this calculation against a numerical fact ( sales, customers) you will run into the problem of how to display the % change in the correct format, since the calculated % will want to take the form of the fact it is calculated against, as can be seen in the example 2 below:


Figure 1 - Pivot Table

Figure 2 - Calculated item

Figure 3 - Results

Not very pretty at all.

As people searched for a work around to this problem 3 common solutions have arisen:
  1. Use HTML formatting tricks to “hide” trigger text in the results, then conditionally format off those triggers. While inventive, as the comments note, this solution falls flat if the report is ever printed, as the PDF engine will pick up and display all of the hidden characters.

  2. Convert the pivot table to a regular table with some complex column formulas. Very time consuming and cumbersome, would also not solve the requirement of showing the dimension values noted as noted in Footnote1.

  3.  Convert the calculated result to text and manually add your formatting characters. I don’t think this actually works since the calculated fields won’t accept logical SQL functions, and this would be very cumbersome.
Now with 11g providing conditional formatting that allows you to override the default data format, this is possible via the following steps:
  1. Add a column that is a COUNT DISTINCT on the dimension that you are calculating across ( in the displayed example, “Time T05 Per Name Year”. This column will serve as your “trigger” to apply your conditional formatting.

    Figure 4 - Column Formula

  2. For each of your facts, apply a conditional format that is triggered when the above column value is zero. In the formatting, apply whatever visual and data formats you desire. In this example we will format the data as a percent, with one decimal place.

    Figure 5 - Condition

    Figure 6 - Format when condition is met
  3. Exclude the “trigger” column from your pivot view. View your results and be satisfied:

    Figure 7 - Correct formatting of calculated item.

* Note that this would also allow you to apply visual formatting if you wanted to distinguish this row/column as a total.

Why it worksThe use of conditional formatting that applies a data type as part of the format is a straightforward leap of logic, but what to use as the trigger? Most people will try to use the dimension they have setup the calculation in. However, if you try to use the text description given to the calculated item you will find that the condition is never applied:

Figure 8 - Condition on dimension
Figure 9 - Condition never met, format never applied
If you try to setup a filter that is true when the dimension is not in reasonable range of values ( in this example we try to format off all years not in the 2000s ) you will find that your calculated item is skipped as well (this has the added vulnerability of being very explicit):

Figure 10 - Condition on dimension values

Figure 11- Condition never met, format never applied

The reason for all this is that the calculated item “borrows” EACH of the dimension values it operates against. Hence, no matter how inventive your filter is, as long as you are trying to somehow separate the calculated member away from the members it is operating on, you will never succeed. This member “borrowing” is apparent if you add the dimension it operates against to the query a 2nd time, and look at the table view.

Figure 12 - Calculated item "borrows" members
But since the “member value” given to the calculated item does not actually exist in the dimension, if you try to perform a count distinct against it, you will always get zero.

Figure 13 - Count distinct against dimension
There is your difference; there is your “trigger.” The rest is basic formatting.

1: You might suggest that this requirement is better served using column(s) with time series calculations, and you might be right. However, more often than not the user will want to SEE the time periods being compared ( 2012 vs 2011, or 08/07/2012 vs 08/07/2011). When using facts with time series calculations you will only be able to show “this year” vs “last year” since the column heading of the time series calculated fact will always be static. In these cases you will need to use the base fact and a time dimension, along with the solution provided here.

2: All screen shots, and examples used in this post are performed in Sample App V305. An XML of the final correctly formatted report can be downloaded here.
Categories: BI & Warehousing

Learn To ______ In A Year

Chet Justice - Tue, 2013-07-30 22:02
It started at The Talent Code blog by Daniel Coyle a few weeks back, What's Your LQ (Learning Quotient)?. That led me to Diamondbacks’ Goldschmidt Has Little Ego and Few Limits. I like baseball stories. I especially like this passage:

“A lot of kids have so much pride that they want to show the coaches and the front office that they know what they’re doing, and they don’t need the help,” Zinter said. “They don’t absorb the information because they want us to think they know it already. Goldy didn’t have an ego. He didn’t have that illusion of knowledge. He’s O.K. with wanting to learn.”

I identify with that. I believe part of my success is because I ask questions.

Back to the original article. Then I end up here, Can Everyone Be Smart at Everything? I seem to lack the ability to focus for extended periods of time. Well, not quite true. I have the ability to focus, but I like to focus on a million different things. Does that count? I don't know.

I'm often envious of my friends who have been DBAs for 20 years, or worked with OBIEE for 10 years (don't argue with me...I know Oracle hasn't owned it for 10 years, I'm looking at you Christian), or APEX for 10 years (that's safe to say). I've flirted with all of those, but I've never committed...See how I get distracted easily? Wow.

And just as importantly, that mistakes are part of good learning. As a Wired article recently reported about why some are more effective at learning from mistakes, “the important part is what happens next.” People with a “growth mindset” — those who “believe that we can get better at almost anything, provided we invest the necessary time and energy” — were significantly better at learning from their mistakes.

and then...

“The meaning of difficulty changes. Difficulty means trying harder, trying a different strategy. They understand that change is possible, and progress occurs over time.”


OMFG. Focus!

Back to the original article and I'm reading through the comments. Someone links up to this young lady who taught herself how to dance in a year. Watch it.



Which finally brings me back to The Talent Code, To Improve Faster, Think Like a Startup. Staying with me? How about this?



Finally, there's a point. I want to do this. Maybe not dance (as much fun as that may be), but something else. Krav Maga? Algebra? Calculus (I'm pursuing my physics or engineering degree in 2035, I need to study my math). I want to test out her technique. Small, discrete steps practiced daily towards some end goal (pass a calc test, take a real estate licensing test, whatever). The problem for me, if you haven't noticed, is focus. This method may help.

If you were to try something like this, what would you set out to learn?
Categories: BI & Warehousing

Technologies behind Oracle Transactional Business Intelligence (OTBI)

Dylan Wan - Thu, 2013-07-25 16:57

Oracle Transactional Business Intelligence (OTBI) is one of the business intelligence solutions provided as part of Fusion Applications. 

To build a real-time BI, the major challenge is to make sure that it can perform and has no or minimum interfere to the core objective of the transactional application, the online processing.

This is the reason why we need Oracle Business Intelligence Applications (OBIA) for Fusion Applications.  The idea is to keep the minimal processing of detecting changes and capturing changes in the transactional system and leave everything else, such as, preparing and consolidating the data for reporting, to BI Applications.

Here are some of the technologies available to make OTBI possible:

1. SQL Trimming from ADF

ADF stands for Application Development Framework.  It is the application development framework used in developing Fusion Applications.  In general, it is a declarative metadata driven framework to let the application developers to define the data model, define the data access layer, define the UI rendering, put the validation logic and processing in the middle tier.

The underlying data model, in most of cases, is still the relational model defined in the Fusion Apps transactional database under the 3rd NF design.

The key enabling technologies provided from ADF to OTBI is the “Composite VO” or “Composite View Object”.  For me, it can generate the database SQL for us based on the metadata.  Unlike the database technology using the database view, ADF engine can look further down to the entity objects included in the view object and selectively choose which entities are needed in a given SQL.  If the view object includes two tables (EOs), one primary EO for data at the line level, and the other EO for getting the parent data, When the query (Composite VO) does not include any column from the parent EO, the SQL generated by ADF will not include the table in the join. 

This is a superior technologies, comparing to the old technologies of building the business views.

If you are a Java programmer and would like to get the feeling about what Composite View Object looks like and how it works, here is a good blog post:

Do you know what is a Composite View Object?

2. BI Platform – ADFQuery to Composite VO

This enabling technology is provided by BI platform and available as a Java library. It adds a layer on top of the ADF composite VO.  Without writing the Java code, it generates the codes of creating the composite VO on the fly.  It allows us to query the data from the ADF engine by sending them a XML block called ADFQuery.

This doc shows some of the ADFQuery XML blocks.

http://docs.oracle.com/cd/E15586_01/fusionapps.1111/e20836/adf.htm#BIEMG3435

To see better examples, you can find them in NQQuery.log files.

It is a query language like SQL.  You have the section  for the column projection, the join criteria using view links, and the filter using view criteria.

Here are other enabling technologies behind OTBI.

3. ADFQuery generation from BI Server

4. SQL By Pass Database

5. Relational to Dimensional Mapping (Physical Layer to Business Model Layer)

6. SELECT Physical in initialization block

7. ADFQuery Initialization block

8. Physical Lookup function from BI platform

9. Logical Lookup function from BI platform

10. Data Security enabled at the VO layer via Fusion AppCore

11. Applcore Tree Flattening

12. Applcore Business Intelligence Column Flatten VO (BICVO)

13. BI Flexfield VO generator

14. BI Extender via Import Wizard

15. BI View Object created based on the BI EE Logical SQL (BIJDBC)

16. Effective Date VO with as of date filter

17. ADF Application Module to BI variable interface

and more…

Regardless, the goal of these technologies is to enable the users to get the real time data access to the Fusion Apps.  There is really little or no much we can do for providing the feature like data snapshot, pre-built aggregate, multiple currencies, data consolidation and conformance, cross subject area analysis, and the most important, the query performance with complexity logic to be available in a reasonable time without the interfere to the transactional system.

Categories: BI & Warehousing

Oracle GoldenGate 11g Handbook

Amardeep Sidhu - Thu, 2013-07-18 10:02

Few months ago I contributed a chapter (on Monitoring, Troubleshooting and Performance tuning) to a GoldenGate book on Oracle Press that Robert Freeman was authoring. Thought of posting a small update that the book is now out. My name doesn’t appear on the main page Sad smile but you will see it in the Acknowledgements section Winking smile Below is a screenshot taken from Amazon preview Smile.

You may want to grab a copy if you are using/planning to use Oracle GoldenGate 11g.

Here is the link to the book page on Amazon. It seems the book is not published in India yet but one can order the imported edition on amazon.in

image

Categories: BI & Warehousing

Exalytics - Version X3-4 is Here

Look Smarter Than You Are - Tue, 2013-07-16 21:15
I've mentioned before that the Exalytics X3-4 was nearly available (the first clue was when it hit the engineered system price list back on June 4).  It was talked about at-length during the Kscope13 Sunday Developer's Symposium and... it's finally here.


Hardware Upgrades
  • RAM.  Doubling from 1 terabyte to 2 terabytes.  This will help everything on the box but those of us running Essbase now have even more RAM to use for making RAM drives.
  • Flash.  Exalytics now comes standard with 2.4 TB of flash.  I mentioned this earlier as an upgrade option to the Exalytics X2-4, but it now is native to the X3-4.  As mentioned in my earlier article, flash impacts Essbase performance far more than OBIEE (which isn't as disk I/O intensive).  Having .25 millisecond read latency (what these flash drives are rated) means there's virtually no seek time finding values in an Essbase cube on disk.  I'm expecting most Essbase customers will put their physical cubes on the flash drives and then quickly load them into a RAM drive upon start up (which has better performance than reading into the Essbase caches for each database).
  • Hard Drive.  They are upping the traditional hard drives from 3.6 TB to 5.4 TB.  It still has 6 physical drives in it, but they are going from 600GB drives to 900GB drives. [Updated on 8-25-2013.]
The cores (still 40) stay the same... for now.  At some point, someone is going to start hitting these limits and they're probably up the cores and I wouldn't be surprised if they went 100% flash drives in a future release.

Software
X3-4 supports OBIEE 11.1.1.7, Endeca 3.0, Essbase 11.1.2.3, and any Linux-allowed Hyperion EPM product on 11.1.2.3.  They also strongly imply that there are some Essbase optimizations in 11.1.2.3 that only work on Exalytics, but I haven't found them yet to verify.  Regardless, Exalytics X3-4 is the best engineered system you can currently buy for Essbase, bar none.

Pricing: $175,000
The X2-4 was $135,000 for the hardware (software sold separately), but to add-on flash, you paid an additional $35,000 giving us a real price for X2-4 of $170,000.  The new box is $175,000... and for that additional $5,000, they double the RAM and increase the hard drives 50%.  In other words, you're getting a hell of a deal.  For what is literally $40,000 more in total, you're getting 1 TB more of RAM, 2.4 TB of really good flash, and 1.8 TB of additional hard drive.

What if I Already Bought an X2-4?
First of all, congratulations.  You're really smart, despite what your high school guidance counselor said.  To upgrade your X2-4 to an X3-4, you can buy an upgrade kit!  The upgrade kit (to get flash and the 1 TB of RAM) does cost $105,000 though.  So your X2-4 with an upgrade to an X3-4 will end up costing you $240,000 in total.  Oracle will support your X2-4 under their lifetime support policy even though it is being phased out.  

Availability
You can order an X3-4 now.  I haven't seen one actually ship yet, but it was just officially launched yesterday.  While I think you can still buy the X2-4 until the end of this quarter (August 31, 2013), I'm not at all sure why you would.  Find the extra $40K and get not only blazingly fast flash drives but more RAM than you know what to do with.
Categories: BI & Warehousing

Next Generation Essbase Outline Extractor Released!

Tim Tow - Sat, 2013-07-13 00:05
After a bunch of work, we have finally released the Next Generation Essbase Outline Extractor. So, what is the Next Generation Essbase Outline Extractor?  Well, the original outline extractor was written in Visual Basic and Visual Basic has has not been supported by Microsoft for a number of years.  In addition, the original Essbase Outline Extractor requires the 32-bit runtime client which is getting harder and harder to support.  In addition, users have asked for new functionality over the years and it just wasn't feasible to add the new functionality to a VB based product.

Some people have asked me "What about the new Maxl Outline Extractor?"  That is a good question, but keep in mind it only works with 11.1.2.x and higher plus it gives you an XML output. You still have to do something with the XML format for it to be useful.

The Next Generation Essbase Outline Extractor was completely redesigned from scratch.  The main design goals were:
  • Design for long term
    • Extensible
    • Use Oracle’s strategic technology
  • Design for supportability
    • Wider adoption
    • Wider platform choices
    • Less support calls
It was also architected for extensibility to enable new functionality to be added and to allow programmers to write their own extensions to the code.  I chose to write the extractor in Java for a number of reasons.  First, it is Oracle's strategic API for the future which means new features of Essbase will be available in the Java API first.  In addition, Java is multi-platform which means it runs on Windows, Linux, AIX, Solaris, and MacOS regardless of whether it is 32-bit or 64-bit.  In addition, with the Java API, we can ship a single zip download that has everything it needs (except Java itself).  There is no need for the runtime client with the new extractor!

The new Outline Extractor also supports a number of features.  Many of these features are ne:
  • Export one or more dimensions in an outline.
  • Filter members to to be exported by dimension.
  • Write to multiple outputs on a single pass of the outline.
  • Output multiple alias tables into a single output file.
  • Write to Hyperion Planning Outline Load Utility format.
  • Output to real relational databases via JDBC.
The extractor was also designed to be run by a batch process using Java properties files.  The properties files can be written from the GUI interface and directly executed from the GUI.  There is even built-in functionality that optionally encrypts the username and password on first usage.

Here are a few screenshots of the new extractor.  The first screenshot shows the GUI screen where you can select one or more writers for output.



The second screenshot shows the option screen showing the options for the Load File Text Writer.

The final screenshot shows the execution screen. 















The Next Generation Outline Extractor is available on our website at www.appliedolap.com.  If you have any questions on the new extractor, don't hesitate to contact us at support@appliedolap.com.

Categories: BI & Warehousing

Write It Out

Chet Justice - Mon, 2013-07-08 10:54
This one was sitting in the drafts folder for a week or two, then I saw this post on Twitter:

I wonder what percentage of people ask a question and figure it out on their own before you help them. What about in your experience?

— Amy Caldwell (@amyccaldwell) July 8, 2013
Years ago I had a boss who was my technical superior (he may still be). I used to pop in and out of his office, or try to, and ask questions. Most of them were silly, n00b questions.

He was nice, but busy. It didn't take me very long to "read" that. So I slowed down my pace of questions. I began to write things up via email so that he could respond when he the had time. I started to use forums as well. Then I found was directed to How To Ask Questions The Smart Way.

One of the things that became evident quickly is that I didn't always have to hit Send (email) or Submit (forum post), just the act of writing it out forced me to think through the issue and more often than not, I would figure out the answer on my own.

Flash forward five or six years and I started to receive all these questions, either in person or via chat. "Send me an email" was usually my response, especially if I was in the middle of something (see: Context Switching). I was happy to help, just not at that moment. With email, I could get to it when I got a break (or needed one).

One of my favorite people, Jason Baer, who has worked for RittmanMead for the last couple of years, took this to heart. We started working together in December of 2009 and he would pepper me with questions constantly. I could never keep up. "Email the question Jason."

I didn't realize it, but I started getting fewer and fewer emails/questions from him. He began to figure them out on his own. It seemed most of the time he had just missed something, other times he just figured out another way to do something.

Jason is a smart guy. I think I'm smart. Sometimes it's just easier to ask the question without thinking it through. In fact, I do that quite a bit on The Twitter Machine ™, especially those errors that I seem to know but just don't have the bandwidth to research (think DBA type questions). I believe the types of questions that should must be written down are those that deal with Approach (design, architecture, etc). Any of those ORA errors better come along with a link to the error code in the documentation and some proof that you've researched it a bit yourself...but then that's getting into How To Ask Questions The Smart Way.

Go out and practice. Next time you have a (technical) question for someone, anyone, write it down and see what happens.
Categories: BI & Warehousing

Major Price Cuts in Essbase, OBIEE, BIFS, and OSSM

Look Smarter Than You Are - Thu, 2013-07-04 19:20
Pricing Went Down 25-40%While Oracle is pretty good at giving discounts off list price, it's rare when they actually cut their list prices.  Shockingly, they just lowered (for what I believe is the first time since these products made it onto the price lists) the per processor prices on several of their Business Intelligence offerings: Essbase, OBIEE (Oracle Business Intelligence Foundation Suite), BIFS (Business Intelligence Foundation Suite), and OSSM (Oracle Scorecard & Strategy Management).

Per the price list dated June 25, 2013, the per processor prices have dropped substantially:

  • Essbase went from $184,000 to $138,000.  That's a 25% decrease.
  • OBIEE went from $295,000 to $221,250.  That's also a 25% decrease.
  • BIFS went from $450,000 to $300,000.  That's a 33% decrease.
  • OSSM went from $149,250 to $89,550.  That's a 40% decrease.
Now think about this for a second.  BIFS (Business Intelligence Foundation Suite) comes with Essbase, OBIEE, OSSM, and a few other fun things like EAL4HFM (Essbase Analytics Link for HFM).  BIFS was already a great deal because just buying Essbase, OBIEE, and OSSM separately was setting you back $628,250 but as a bundle costs you only $450,000.  That's a 28% decrease off just those 3 components separately.  Now those separate components list at $448,800 or if you buy the BIFS bundle, $300,000 which is a 33% discount off the components separately.

In other words, you now get OBIEE, Essbase, OSSM, and some other products for just $5,000 more per processor than OBIEE cost alone 2 weeks ago (it was $295,000, remember).  The named user costs for these products has not changed which means that they are positioning these price cuts directly at the enterprise customers: companies who are looking to adopt Oracle Business Analytics across their organization.  Considering those prices above are list, enterprise customers should be getting a discount starting off those prices which makes processor licensing start to seem very attractive for large deployments.

Core Factors
Also remember that Oracle doesn't charge this full price for every core on the processor.  They have a "processor factor" which charges less per core.  Depending on the type of processor, the Oracle Processor Core Factor Table will charge between 25% and 100% of the list processor price.

Take Exalytics X2-4, for example.  It has 4 Intel Xeon E7-4800 chips in it.  Each of those chips has 10 cores giving you 40 cores in total.  Based on the Processor Factor, these cores count as only half a processor.  In other words, to license a full X2-4, you'd need to pay for 20 processor licenses which at the new $300,000 price means a list of $6,000,000.  That's the maximum (not including tax, maintenance, TimesTen, etc.) that you'd pay but it would assumedly come in less than that which is really impressive to license an entire Exalytics box for unlimited users.  Unlimited, people.  Your whole organization could access OBIEE and Essbase for at most $6MM in software.

This may be the pricing discount your company needs to buy unlimited user licenses of Oracle Business Analytics.  And don't hold your breath for Oracle to drop any more list prices.  Take it as a gift and buy it before they change their minds.

Update as of 7-15-13According to an article on Information Week, during the release of Exalytics X3-4, Paul Rodwick was asked about the recent price decreases mentioned above.  He gave the intriguing response that while the prices did go down, it's "old news" because Oracle stealthily did it 9 months ago.  While I don't have the technology price list he's referring to (if you do, post a link to it in the comments), here's Paul's quote:
The cost for BI Foundation Suite on a named-user basis has never been changed, but about nine months ago we adjusted per-CPU pricing in part because we were seeing more customers want to license the full complement of Exalytics.
Categories: BI & Warehousing

Context Switching: An Example

Chet Justice - Mon, 2013-07-01 15:45
Last week at #kscope13 I saw an outstanding example of context switching. If you don't know what it is, Tom Kyte explains it here.

The two environments are just "different", separate and distinct. You can do plsql without SQL, you can do SQL (and many times do) without invoking plsql. There is a call overhead to go from SQL to PLSQL (the "hit" is most evident when SQL invokes PLSQL - not so much the other way, when SQL is embedded in PLSQL). Even if this hit is very very small (say 1/1000th of a second) - if you do it enough, it adds up. So, if it can be avoided - it should be.

The session was Using Kanban and Scrum to Increase Your Development Throughput presented by Stew Stryker (not to be confused with Ted Striker) of Dartmouth College (Stew gave me a gallon of Vermont Maple Syrup which exploded in my bag on the flight home, a gift for sharing my hotel room. Thanks Stew! ;)). So here's the example he gave to demonstrate context switching.

Take a list of names and time yourself writing out the first letter of each name, then the second, until you are finished.



Now, same list of names and write them out the way you normally would, left to right.



If the first method was faster, you are a freak of nature.
Categories: BI & Warehousing

Oracle database 12c

Amardeep Sidhu - Wed, 2013-06-26 20:58

So there is a new toy in the market for database geeks : Oracle has released database 12c. Every social platform is abuzz with the 12c activity. So thought that I should also complete the ritual Winking smile

In this post Aman has already summed up many important links.

Maria Colgan has posted some useful links here.

And here is a link to a slidedeck about Upgrading and Migrating to 12c.

Happy 12c’ing !

Categories: BI & Warehousing

Kscope - Oracle Business Analytics Strategy & New Features

Look Smarter Than You Are - Sun, 2013-06-23 11:34
"Business Analytics is a key strategic priority for Oracle."
                 - Paul Rodwick
I'm sitting in the Kscope13 BI Symposium listening to keynote speaker Paul Rodwick, VP of Oracle BI Product Management. Paul was rather interesting despite his flight having landed in New Orleans at 4AM.  On 3-4 hours sleep, Paul reviewed Oracle's Business Analytics strategy.  It's surprising to me how little Oracle's EPM/BI architecture has changed over the last 5 years (other than the renaming to "Oracle Business Analytics."  This is a good thing.
Why?  Because over the last 5 years, the architecture has gone from a products-integrating-is-a-theoretically-good-idea-so-let's-put-it-on-a-slide-cross-our-fingers-and-see-what-happens to an actual integrated solution that uses the various products in the Oracle Business Analytics line together with each product doing a key part.  Instead of "Essbase or OBIEE or an application?" it's "Essbase as the cube platform, OBIEE as the front-end, applications for needs that are often common across multiple companies."

So now that Oracle has gotten the basics out of the way, they're looking to expand their Business Analytics offerings.  Their key focuses for the immediate future are big data, mobile, in-memory computing, and cloud-based analytics.  The last two really speak to technology of deployment (in-memory and cloud), big data seems to be one of those things that everyone is talking about and no one's quite sure what to do with for the moment, but mobile is on everyone's minds and people are actually doing something about it.  To further that immediate mobile need, Oracle is releasing new functionality in every release or patch of the Oracle mobile analytics products.  For instance, Oracle 11.1.1.7 now has a full mobile security toolkit (available on OTN) for companies that want greater security than native Apple iOS provides.

Paul discussed some of the key features in the 11.1.1.7 release (including Smart View as the primary Office front-end for BI going forward).  He mentioned that the bundled patch for OBIEE 11.1.1.7 will be out on a few weeks, so prep yourself for 11.1.1.7.1.  He also talked about some recent improvements to Endeca in version 3.0 of that product.  While I love Endeca's extremely powerful ability to discover information in unstructured data, right now, most companies are still focused on analyzing their structured information.  Unstructured analysis is definitely coming: it's just only being deployed by a handful of leading-edge companies at the moment.
Where Are They Going?The key releases we should see in the next 9-12 months will revolve around these themes:
  • Visual analysis.  They're trying to make the analysis more intuitive because the majority of users don't spend their day being analysts: they want the system to help them find issues quickly so they can make better business decisions faster.
  • Mobile Analytics. Oracle is planning to create a BI Mobile "Applications Designer" that will allow developers to make HTML5 applications purpose-built for mobile deployment.  They will also continue to improve the mobile applications every version but they didn't go into what some of the new improvements are going to be specifically beyond more HTML5 deployment.
  • Exalytics.  They promised a new Exalytics announcement in the near future.  I'm presuming this refers to the new Exalytics X3-4 version that's mentioned on the June 4 Oracle Engineered System Price List (page 5).  I expect this will be detailed more during Steve Liebermensch's session later this week.
  • Cloud analytics.  Oracle is making a huge investment in the cloud and it looks like there will be more and more applications in Oracle Business Analytics that run in the cloud.  This makes it a lot easier for customers to get immediate ROI from a BI implementation without huge server investments.
  • Big data.  Part of Oracle's strategy in this area is to tie into any data in any source behind the scenes into Oracle BI.  Data agnostic
  • Predictive analytics.  Paul didn't really talk to this one other than to tease that they do have dedicated resources to expanding the Predictive Analytics capabilities of Oracle BI Foundation Suite.  There is some P.A. functionality in Hyperion Planning, Crystal Ball, and Hyperion Strategic Finance and that sounds like it will be expanded into the BI layer in future releases.
The one thing that's really apparent from Paul's session is that Business Analytics is now a $1+ billion dollar portion of Oracle revenue... and they're treating it as such in terms of research and development.  It's a fast growing space and Oracle seems determined to maintain their market share in overall Business Analytics.

I hope to blog later in the week if any new announcements come out.  Coming to you from Kscope13, this is your humble reporter, Edward Roske.
Categories: BI & Warehousing

agent deployment error in EM 12c

Amardeep Sidhu - Sun, 2013-06-16 12:04

Yesterday I was configuring EM 12c for a Sun Super Cluster system. There were a total of 4 LDOMs where I needed to deploy the agent (Setup –> Add targets –> Add targets manually). Out of these 4 everything went fine for 2 LDOMs but for the other two it failed with an error message. It didn’t give much details on the EM screen but rather gave a message to try to secure/start the agent manually. When I tried to do that manually the secure agent part worked fine but the start agent command failed with the following error message:

oracle@app1:~$emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
Starting agent ………………………………………………………. failed.
HTTP Listener failed at Startup
Possible port conflict on port(3872): Retrying the operation…
Failed to start the agent after 1 attempts.  Please check that the port(3872) is available.

I thought that there was something wrong with the port thing so I cleaned the agent installation, made sure that the port wasn’t being used and did the agent deployment again. This time it again failed with the same message but it reported a different port number ie 1830 agent port no:

oracle@app1:~$emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
Starting agent ……………………………………………. failed.
HTTP Listener failed at Startup
Possible port conflict on port(1830): Retrying the operation…
Failed to start the agent after 1 attempts.  Please check that the port(1830) is available.

Again checked few things but found nothing wrong. All the LDOMs had similar configuration so what worked for the other two should have worked for these two also.

Before starting with the installation I had noted the LDOM hostnames and IPs in a notepad file and had swapped the IPs of two LDOMs (actually these two only Smile with tongue out ). But later on I found that and corrected. While looking at the notepad file it occurred to me that the same stuff could be wrong in /etc/hosts of the server where EM is deployed. Oh boy that is what it was. While making the entries in /etc/hosts of EM server, I copied it from the notepad and the wrong entries got copied. The IPs for these two LDOMs got swapped with each other and that was causing the whole problem.

deinstalled the agent, correct the /etc/hosts and tried to deploy again…all worked well !

Categories: BI & Warehousing

Required Reading

Chet Justice - Wed, 2013-06-12 13:10
It's not often that I run across articles that really resonate with me. Last night was one of those rare occasions. What follows is a sampling of what I consider to be required reading for any IT professional with a slant towards database development. Bad CaRMa

how NOT to design a database schema - super classic article. Every data architect should read this ! simple-talk.com/opinion/opinio… @timothyjgorman

— Kyle Hailey (@dboptimizer) June 11, 2013
That led me to Bad CaRMa by Tim Gorman. This was an entry in Oracle Insights: Tales of the Oak Table, which I have not read, yet.

A snippet:

...The basic premise was that just about all of the features of the relational database were eschewed, and instead it was used like a filing system for great big plastic bags of data. Why bother with other containers for the data—just jam it into a generic black plastic garbage bag. If all of those bags full of different types of data all look the same and are heaped into the same pile, don't worry! We'll be able to differentiate the data after we pull it off the big pile and look inside.

Amazingly, Randy and his crew thought this was incredibly clever. Database engineer after database engineer were struck dumb by the realization of what Vision was doing, but the builders of the one-table database were blissfully aware that they were ushering in a new dawn in database design...

This is from 2006 (the book was published in 2004). Not sure how I missed that story, but I did. Big Ball of Mud I've read this one, and sent it out, many times over the years. I can't remember when I first encountered it, but I read this once every couple of months. I send it out to colleagues about as often. You can find the article here.

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition. Programmers with a shred of architectural sensibility shun these quagmires. Only those who are unconcerned about architecture, and, perhaps, are comfortable with the inertia of the day-to-day chore of patching the holes in these failing dikes, are content to work on such systems.

Read it. Remember it. How To Ask Questions The Smart WayEver been in a forum? Has anyone ever given you the "RTFM" answer? Here's how you can avoid it. How To Ask Questions The Smart Way. I read this originally about 9 or 10 years ago. I've sent it out countless times.

The first thing to understand is that hackers actually like hard problems and good, thought-provoking questions about them. If we didn't, we wouldn't be here. If you give us an interesting question to chew on we'll be grateful to you; good questions are a stimulus and a gift. Good questions help us develop our understanding, and often reveal problems we might not have noticed or thought about otherwise. Among hackers, “Good question!” is a strong and sincere compliment.

Despite this, hackers have a reputation for meeting simple questions with what looks like hostility or arrogance. It sometimes looks like we're reflexively rude to newbies and the ignorant. But this isn't really true.

What we are, unapologetically, is hostile to people who seem to be unwilling to think or to do their own homework before asking questions. People like that are time sinks — they take without giving back, and they waste time we could have spent on another question more interesting and another person more worthy of an answer. We call people like this “losers” (and for historical reasons we sometimes spell it “lusers”).
Business Logic - PL/SQL Vs Java - Reg The article can be found here.

I'm don't believe this is the one that I would read just about every day during my first few years working with Oracle, but it's representative (I'll link up the original when I find it). I cut my teeth in the Oracle world by reading AskTom every single day for years. Some of my work at the time included working with java server pages (jsp) - at least until I found APEX. I monkeyed around with BC4J for awhile as well, but I believe these types of threads on AskTom kept me from going off the cliff. In fact, I got to a point where I would go to an interview and then debate the interviewer about this same topic. Fun times.

if it touches data -- plsql.

If it is computing a fourier transformation -- java.

If it is processing data -- plsql.

If it is generating a graph -- java.

If it is doing a transaction of any size, shape or form against data -- plsql.
Thinking Clearly About Performance Cary Millsap. Most of the people seem to know Cary from Optimizing Oracle Performance, I didn't. I first "met" Cary virtually and he was gracious enough to help me understand my questions around Logging, Debugging, Instrumentation and Profiling. Anyway, what I've learned over that time, is that Cary doesn't think of himself as a DBA, he's a Developer. That was shocking for me to hear...I wonder how many others know that. So I've read this paper about 20 times over the last couple of years (mostly because I'm a little slow). I organize events around this topic (instrumentation, writing better software, etc) and this fits in perfectly. My goal is to one day co-present with Cary, while playing catch, on this topic (I don't think he knows that, so don't tell him). Link to his paper can be found here. Enjoy! The Complicator's Gloves One of my favorite articles from The Daily WTF of all time. Find the article here. The gist of the story is this: an internal forum where people were discussing how to warm a given individuals hands on his bike ride to work. The engineers then proceeded to come up with all kinds of solutions...they spent all day doing this. Finally, someone posts, "wear gloves." End of discussion. Love it. I wrote about it years ago in Keeping it Simple. For a few years I considered buying up thecomplicatorsgloves.com and try to gather related stories, but I got lazy. You should read this often, or better, send it out to colleagues on a regular basis to remind them of their craziness.

I'll continue to add to this list as time goes on. If you have any suggestions, leave a comment and I'll add them to the list.
Categories: BI & Warehousing

Looking Forward to Kscope13

Look Smarter Than You Are - Thu, 2013-06-06 23:17
On June 9, the rates for Kscope13 go up $300 per person (basically, you're going up to the last minute, I-don't-know-why-I-waited-but-now-it-costs-a-lot-more price).  If you haven't registered yet for what is by far the best Oracle EPM, BI, Hyperion, Business Analytics, Essbase, etc. conference in the world, go right now to kscope13.com and register.  It'll be the best training experience of the year: you're basically getting 4.5 days of training that you won't see anywhere else the entire year... for the price of 2 days of training at an Oracle training center.

And when you register, don't forget to use promo code IRC to save $100 off whatever the current rate is.

The conference is June 23-27 in New Orleans though my favorite day is always the opening Sunday, so make sure you fly in Saturday night.  On Sunday, they turn the sessions over to the Oracle Development team to talk about everything they have planned for the next 1-3 years.  It's the one time each year that you can hear right from the people who are building it what you're going to be seeing in the future.  There's generally an hour on each major product line (an hour on Essbase, an hour on Hyperion Planning, an hour on mobile BI, etc.).  The keynote this year is Balaji Yelamanchili, the head of Oracle BI and EPM development for Oracle.  My only semi-complaint about this year's BI/EPM Symposium is that there's so much content that they're splitting it into three concurrent symposiums: Business Intelligence, EPM, and a special symposium for the EPM business users.

This year will be somewhat bittersweet for me since I am no longer actively involved with the chairing of the conference.  This means that I get to focus on going to sessions, learning things, playing/leading Werewolf games, and of course, presenting a few sessions.  Here are the ones I'm personally teaching:


  • Using OBIEE to Retrieve Essbase Data:  The 7 Steps You Won’t Find Written Down.  This is in the BI track and it's basically all the quirks about connecting OBIEE to Essbase in a way that uses the strengths of each product.
  • What’s New in OBIEE 11.1.1.7: Oracle on Your iPhone & Other Cool Things.  This is also in the BI track and it's an overview of all the things that people will like in 11.1.1.6 (for both Hyperion and relational audiences).
  • Everything You Know About Essbase Optimization  is Incomplete, Outdated, Or Just Plain Wrong.  This is in the Essbase track and it's the one I'm most looking forward to delivering, because I get to break all of the optimization rules we all have been accepting as gospel for close to 20 years.
  • Learn From Common Mistakes: Pitfalls to Avoid In Your Hyperion Planning Implementation.  This is a vendor presentation hosted by interRel.  I get to sit on the panel and answer Planning questions from the audience while talking about blunders I've seen during Planning implementations.  It should be fun/rousing.  Since it's all interRel, I wouldn't be surprised if a few punches were thrown or at minimum, a few HR violations were issued.
  • Innovations in BI:  Oracle Business Intelligence against Essbase & Relational (parts 1 and 2).  This is also in the BI track (somehow I became a BI speaker???) and I'm co-presenting this session with Stewart Bryson from Rittman Mead.  We'll be going over OBIEE on Essbase on relational and compare it to OBIEE on relational directly.  Stewart is a long-time friend and Oracle ACE for OBIEE, so it should let us each showcase our respective experiences with Essbase and OBIEE in a completely non-marketing way.
  • CRUX (CRUD meets UX): Oracle Fusion Applications Functional UI Design Patterns in Oracle ADF.  This is in the Fusion track and I'll be talking about how to make a good user interface as part of the user experience of ADF.  No, this doesn't have a thing to do with Hyperion.
I am looking forward to all the wacky, new things Mike Riley (my replacement as Conference Chair for Kscope) has in store.  My first Kscope conference was in New Orleans in 2008 (back when they called it Kaleidoscope and no one was quite sure why it wasn't "i before e") so this is a homecoming of sorts albeit with 8 times as many sessions on Oracle BI/EPM.  If you're there (and let's face it, all the cool kids will be), stop by the interRel booth and say "hi."  It's the only 400 square feet booth, so it shouldn't be hard to find.
Categories: BI & Warehousing

Webcast Series - What's New in EPM 11.1.2.3 and OBIEE 11.1.1.7

Look Smarter Than You Are - Tue, 2013-06-04 10:56
Today I'm giving the first presentation in a 9-week long series on all the new things in Oracle EPM Hyperion 11.1.2.3 and OBIEE 11.1.1.7.  The session today (and again on Thursday) is an overview of everything new in all the products.  It's 108 slides which goes to show you that there's a lot new in 11.1.2.3.  I won't make it through all 108 slides but I will cover the highlights.

I'm actually doing 4 of the 9 weeks (and maybe 5, if I can swing it).  Here's the complete lineup in case you're interested in joining:

  • June 4 & 6 - Overview
  • June 11 & 13 - HFM
  • June 18 & 20 - Financial Close Suite
  • July 9 & 11 - Essbase and OBIEE
  • July 16 & 18 - Planning
  • July 23 & 25 - Smart View and Financial Reporting
  • July 30 & Aug 1 - Data & Metadata Tools (FDM, DRM, etc.)
  • Aug 6 & 8 - Free Supporting Tools (LCM, Calc Mgr, etc.)
  • Aug 13 & 15 - Documentation

If you want to sign up, visit http://www.interrel.com/educations/webcasts.  There's no charge and I don't do marketing during the sessions (seriously, I generally forget to explain what company I work for).  It's a lot of information, but we do spread it out over 9 weeks, so it's not information overload.

And bonus: you get to hear my monotone muppet voice for an hour each week. #WorstBonusEver
Categories: BI & Warehousing

Using Load Plan for managing your ETL task in BI Apps 11.1.1.7.1 (1)

Dylan Wan - Tue, 2013-05-28 02:24

One of the major change introduced in BI Apps 11.1.1.7.1 is the way how we manage the ETL task sequence and trim the unnecessary tasks.

This functionality was accomplished earlier using DAC.  The problem we frequently faced was that the DAC repository and the INFA repository are maintained as two separate repositories.  We have to sync up the name of tasks exactly in order to use DAC to manage the task execution of the Informatica workflow tasks.

Load Plan and Load Plan Generator was designed for addressing this requirement.

Here is a good article that describes the story.

Load Plan Generator – An Inside Look
Categories: BI & Warehousing

Exalytics - Now with 2.4 Tb of Flash

Look Smarter Than You Are - Mon, 2013-05-13 22:26
I'm not sure why there wasn't a major announcement about this, but as of April 9, customers buying an Exalytics machine to speed up their Oracle Business Intelligence can get 2.4 Tb of PCIe flash drives from Oracle certified and engineered to run on Exalytics.  The cost (as of April 9's price list) is $35,000 (search for "flash upgrade kit").

While I haven't seen one in action yet, the flash pack seems to be 6 Sun Flash Accelerator F40 PCIe Cards each of which has a capacity of 400 Gb.  These cards run amazingly fast with read times of more than 2 GB/second (write time is about half that speed at 1+ GB/second).  These cards normally sell for almost $6K each, so Oracle is providing the flash add-on pack for no more markup than you'd get if you bought them on your own (but you'd then have to get them into the Exalytics machine all on your own).
This Matters If You Own EssbaseWhy would you want this?  Essbase, primarily.  Essbase uses a ton of disk I/O and one of the ways Exalytics can speed up Essbase is by pulling your cubes into a RAMDisk (since you have 1 Tb of RAM to play with).  At some point, though, it has to get that data from physical drives to a RAMDisk (unless you're building all your cubes at start up in memory each time).  Having blazingly speedy flash drives with .25 millisecond read latency allows you to store your cubes on the flash drive and then pull into RAM much more quickly than reading from traditional drives.

We have tested Essbase running on flash drives and it helps everything (particularly minimizes the negative effects of fragmentation since seek time drops to basically nothing on flash).  For customers buying Exalytics primarily for Essbase, the Exalytics Flash Upgrade Kit should be strongly considered with every Exalytics purchase (and if you already own Exalytics, buy it to put on top).

OBIEE is much less affected by hard drives, so while it may help OBIEE, this really matters a lot more to Essbase customers.
Oracle EPM Fully Supported on ExalyticsSince we're on the subject of Exalytics, now that 11.1.2.3 is out, all Oracle EPM/Hyperion components certified to run on Linux will run on Exalytics PS2.  These include:

  • Administration Services
  • Calculation Manager
  • EPM Workspace
  • Essbase Server
  • Essbase Studio Server
  • Financial Reporting
  • Interactive Reporting (32-bit only)
  • Oracle HTTP Server
  • Planning
  • Profitability and Cost Management
  • Production Reporting (32-bit only)
  • Provider Services
  • Reporting and Analysis Framework Services and Web Application 
  • Shared Services
  • Web Analysis
Categories: BI & Warehousing

BI Apps 11.1.1.7.1 (BI Apps in ODI) is available in OTN

Dylan Wan - Thu, 2013-05-09 14:18
First Go to OTNwww.oracle.com/technetworkGo to DOWNLOADS -> MiddlewareYou will see Business Intelligence Applications under MiddlewareYou see both 10g and 11g. Click on Oracle Business Intelligence 11g downloadshttp://www.oracle.com/technetwork/middleware/bi-enterprise-edition/downloads/bus-intelligence-11g-165436.htmlYou will see BI Apps 11.1.1.7.1 here:Oracle Business Intelligence Applications (11.1.1.7.1)
Categories: BI & Warehousing

Hyperion 11.1.1.x Drops Off Full Support in July 2013

Look Smarter Than You Are - Mon, 2013-05-06 22:50
Someone recently told me that they just upgraded their Hyperion applications to 11.1.1.4.  I asked them why they didn't go to 11.1.2.  They gave me the standard story about wanting to wait until the current version got stable (even though 11.1.2.0 came out over 3 years ago in April of 2010).

What they didn't know (cause apparently it hasn't been communicated well) is that Hyperion 11.1.1.x support drops from Premier Support to Sustaining Support in July of 2013 (only two months from the time I'm writing this).  For anyone who doesn't know, Sustaining Support is equivalent to life support.  While Oracle's Lifetime Support Policy does say that you can stay on versions of Oracle's products indefinitely, they don't agree to fully support them.

At the Premier Support level (the one all products start on), you get all the support you'd expect.  When you call in, the support people help you find the bug, they patch it in the next release, you install it, and life goes on happily.  Also, as new versions of supporting products come out like new versions of Office, Windows, or your web browser, Premier Support will make sure the Oracle products work with these new versions.

Extended Support (if offered at all for your Oracle product) comes about 5 years after a product is released.  At this point, Oracle will still let you do all the Premier Support things, they'll just charge you a premium for doing so.  Extended Support will not be offered on Hyperion 11.1.1.x (there aren't enough customers to warrant it).

Sustaining Support (AKA "life support") allows you to call in to ask for support.  Oracle will help you with questions, look up your problems in their knowledgebase, and help you troubleshoot.  They won't patch anything, make versions available that are compatible with new releases of Windows, Office, etc., and in general do anything beyond the bare minimum required.  From Oracle's Lifetime Support Policy document from March 2013, here's what Sustaining Support doesn't do:
  • New updates, fixes, security alerts, data fixes, and critical patch updates
  • New tax, legal, and regulatory updates
  • New upgrade scripts
  • Certification with new third-party products/versions
  • Certification with new Oracle products
And Hyperion (Oracle EPM) 11.1.1.x falls to this level in July 2013. To point out the obvious, if you're not already planning an upgrade to 11.1.2.x, you should start now.  You really don't want to be on Sustaining Support for long.  Since 11.1.2.3 just was released last week, I would probably hesitate going to that version until the first patch set update is released (probably this summer). If you're doing your upgrade before then, I would suggest going to 11.1.2.2.xxx (the current patch of 11.1.2.2).

If you want to read the sunset dates (the dates they drop to Sustaining Support) for all the current releases, visit Oracle.com for the current Lifetime Support Policy.  Here's the one from March 2013 (scroll to page 22) for the Hyperion products:
http://www.oracle.com/us/support/library/lifetime-support-applications-069216.pdf
Categories: BI & Warehousing

Pages

Subscribe to Oracle FAQ aggregator - BI & Warehousing