BI & Warehousing

Recap Post

Marcos Campos - Thu, 2007-12-06 12:11
For the past couple of months the blog took a back seat. Basically, since KDD, I have had very little time to write. I have been on the road quite a bit and my trip to KDD unleashed a number of research ideas that I have been following up. I will post on the latter over time as the results mature.I have also dropped the ball answering many of the emails and comments I have received. I have caughtMarcos
Categories: BI & Warehousing

Oracle BI Applications and Embedded BI, Part II

Dylan Wan - Wed, 2007-12-05 13:04

This is a topic I wrote in six month ago. In the Part I of this series ofarticles, I mentioned that a warehouse like architecture is required ina heterogeneous environment. I want to elaborate more about this. Inthe future posts, I will also describe the integration technology Ilearned for supporting the embedded BI.

Read the rest of this entry >>

Categories: BI & Warehousing

Key Roles involved in a BI Data Warehouse Project

Dylan Wan - Fri, 2007-11-30 14:10

To develop ordeploy a BI solution for your organizations, you need to have the rightpeople involved in the time time. Here are typical roles involved in aBI data warehouse project.

  • Project Sponsor
  • Project Manager
  • Functional Analyst
  • SME
  • BI Architect
  • ETL Developers
  • DBA

The job description and responsibilities are listed in this table: Read the rest of this entry >>

Categories: BI & Warehousing

DSS and BI

Dylan Wan - Fri, 2007-11-30 14:09

I found a very old book, called Decision Support Systems: An Organizational Perspective, in a library last weekend. It was written by Peter Keen,an author of several popular books, which help many business managersand users understand the value of information technology. His DSS bookdraw my attention because he is also the author of my textbook Network in Actions.

More...The DSS book uses a very typical and conventional categorization system which puts the IT systems into three types:

Transactional System, Structure Decision system, and Decision Support System.

These categories are created based on the classification ofdecisions into structured, unstructured, and partially structureddecision. His focus is the 3rd category, DSS. Peter believes that a DSSshould assist in solving the semi-structured problems. A DSS shouldsupport, not replace, the managers.

I feel that the above is a very good framework to view the role ofan analytics apps. A BI analytics application should be a DSS solution.However, BI analytics apps can do much more then just a decisionsupport system. BI may help the structured decision making.

BI is not just a collection of reports. The design of a BI analyticsapps needs to consider what are the business decision need to make andwhat kind of information is helpful for making the decision.

Categories: BI & Warehousing

Data Warehouse Project Lifecycle

Dylan Wan - Fri, 2007-11-02 01:39

Here is the typical lifecycle for data warehouse deployment project:

0. Project Scoping and Planning

Project Triangle - Scope, Time and Resource.

  • Determine the scope of the project - what you would like toaccomplish? This can be defined by questions to be answered. The numberof logical star and number of the OLTP sources
  • Time - What is the target date for the system to be available to the users
  • Resource - What is our budget? What is the role and profile requirement of the resources needed to make this happen.

1. Requirement

  • What are the business questions? How does the answers of these questions can change the business decision or trigger actions.
  • What are the role of the users? How often do they use the system?Do they do any interactive reporting or just view the defined reportsin guided navigation?
  • How do you measure? What are the metrics?

2. Front-End Design

  • The front end design needs for both interactive analysis and the designed analytics workflow.
  • How does the user interact with the system?
  • What are their analysis process?

3. Warehouse Schema Design

  • Dimensional modeling - define the dimensions and fact and define the grain of each star schema.
  • Define the physical schema - depending on the technology decision.If you use the relational tecknology, design the database tables

4. OLTP to data warehouse mapping

  • Logical mapping - table to table and column to column mapping. Also define the transformation rules
  • You may need to perform OLTP data profiling. How often the data changes? What are the data distribution?
  • ETL Design -include data staging and the detail ETL process flow.

5. Implementation

  • Create the warehouse and ETL staging schema
  • Develop the ETL programs
  • Create the logical to physical mapping in the repository
  • Build the end user dashboard and reports

6. Deployment

  • Install the Analytics reporting and the ETL tools.
  • Specific Setup and Configuration for OLTP, ETL, and data warehouse.
  • Sizing of the system and database
  • Performance Tuning and Optimization

7. Management and Maintenance of the system

  • Ongoing support of the end-users, including security, training, and enhancing the system.
  • You need to monitor the growth of the data.

Categories: BI & Warehousing

KDD 2007

Marcos Campos - Sun, 2007-08-12 08:34
For the next couple of days I am going to be attending the KDD (Knowledge Discovery in Databases) 2007 conference (conference website) along with some other Oracle colleagues. KDD is one of the primary conferences on data mining. This year it will take place in San Jose, CA, from August 12 to 15.Oracle is a Gold sponsor for the event and will have a large presence at the conference. Among other Marcos
Categories: BI & Warehousing

On the Road and Upcoming Talks

Marcos Campos - Mon, 2007-07-09 20:51
This week I am going to be in San Francisco. I have been invited to give a talk at the San Francisco Bay ACM Data Mining SIG on Wednesday. The title of the talk is In-Database Analytics: A Disruptive Technology. Here is a link with information on the talk.On Friday morning, I am presenting at the ST Seminar at Oracle's headquarter. The title of that talk is In-Database Mining: The I in BI. If Marcos
Categories: BI & Warehousing

Way Cooler: PCA and Visualization - Linear Algebra in the Oracle Database 2

Marcos Campos - Mon, 2007-06-04 12:05
This post shows how to implement Principal Components Analysis (PCA) with the UTL_NLA package. It covers some of the uses of PCA for data reduction and visualization with a series of examples. It also provides details on how to build attribute maps and chromaticity diagrams, two powerful visualization techniques.This is the second post in a series on how to do Linear Algebra in the Oracle Marcos
Categories: BI & Warehousing

Webcast Announcement: Oracle's In-Database Statistics

Marcos Campos - Wed, 2007-05-02 00:11
Today (Wednesday), May 2, 2007 at 12:00 PM EST, the Oracle Business Intelligence, Warehouse and Analytics (BIWA) Special Interest Group (SIG) will host another interesting free webcast:Oracle's In-Database StatisticsSpeaker: Charlie BergerSession AbstractOracle Database 10g embeds a range of SQL-based basic statistical functions including: summary statistics, hypothesis testing, correlation Marcos
Categories: BI & Warehousing

Webcast Announcement: A Simple Fraud Detection Application using Oracle Data Mining, SQL Developer and Oracle BI EE

Marcos Campos - Tue, 2007-04-24 12:18
Tomorrow, April 25, 2007On April 25, 2007 at 11:45 AM EDT, the Oracle Business Intelligence, Warehouse and Analytics (BIWA) Special Interest Group (SIG) will host the following free webcast:A Simple Fraud Detection Application using ODM, BIEE, and SQL DeveloperSpeaker: Bob HaberstrohSession AbstractClassification is an often-used methodology in data mining that creates a predictive model Marcos
Categories: BI & Warehousing

Way Cool: Linear Algebra in the Oracle Database 1

Marcos Campos - Fri, 2007-04-20 15:32
New to the Oracle Database 10g Release 2 is a hidden gem, the UTL_NLA package. This not very well known package (you don't get many hits for it in Google) brings linear algebra functionality to the Oracle Database. It makes the Oracle Database an even better platform for scientific and advanced analytics programming. Now it is possible to write performant matrix code in the database easily and Marcos
Categories: BI & Warehousing

Wikipedia and Oracle Data Mining

Marcos Campos - Mon, 2007-02-12 15:44
Wikipedia has a nice page on Oracle Data Mining (link). It provides a good overview of the features and history of the product. Here is a snippet of the text:Oracle Data Mining (ODM) is a software product distributed as an option to Oracle Corporation's Relational Database Management System (RDBMS) Enterprise Edition (EE). This product supports a collection of data mining and data analysis Marcos
Categories: BI & Warehousing

New Oracle Statistical Functions Page

Marcos Campos - Mon, 2007-02-12 14:55
OTN has a new page (link) describing the statistical functions in the Oracle 10g Database. These functions are available in all versions of the database at no extra cost. Features include:Descriptive statisticsHypothesis testingCorrelations analysis (parametric and nonparametric)Ranking functionsCross Tabulations with Chi-square statisticsLinear regressionANOVATest Distribution fitWindow Marcos
Categories: BI & Warehousing

Welcome BIWA

Marcos Campos - Mon, 2007-02-12 14:37
The Business Intelligence, Warehousing and Analytics Special Interest Group (BIWA SIG, BIWA for short) has been recently created. Although it counts with a strong participation of Oracle employees, BIWA is an independent organization from Oracle.BIWA is a community in the making. It provides a number of benefits to its members (membership is free):Get the latest information about Business Marcos
Categories: BI & Warehousing

Merry Christmas, Happy New Year, and a Poll

Marcos Campos - Sun, 2006-12-24 04:30
It has been a great year. My daughter was born as well as this blog. I have launched this blog at the beginning of the year (January first to be more precise) and the readership has been great. Amongst the posts, Time Series and Automatic Pivoting were probably the most viewed. I am on vacation in Brazil right now enjoying a family reunion. I have a big family and it is hard to get everyone Marcos
Categories: BI & Warehousing

Announcement: Oracle Data Mining Consultants Partnership Program

Marcos Campos - Fri, 2006-12-15 10:03
We're starting a program to work with qualified data mining consultants.You and your colleagues are invited to participate in a 2 day hands-on session designed for data mining consultants here in the Oracle Burlington MA office February 7 & 8, 2007. It is also possible to attend remotely via webminar. Space is limited, so please RSVP asap.The Oracle Data Mining Consultants Partnership Program hasMarcos
Categories: BI & Warehousing

Free Webinar: Competing on Analytics

Marcos Campos - Tue, 2006-10-31 06:37
I blogged some time ago (link) about an article on The Harvard Business Review by Babson College's Tom H. Davenport on how analytics are becoming a key competitive factor for companies. I have just learned that Prof. Davenport is giving a free webinar today. The theme is "Competing on Analytics." What participants will learn:What data-driven marketing is (and isn't)How marketing visionaries like Marcos
Categories: BI & Warehousing

Oracle Data Mining in Argentina

Marcos Campos - Mon, 2006-10-30 19:22
I spent the week of the 18th in Buenos Aires spreading the word on Oracle Data Mining. I was invited by Snoop Consulting as a keynote speaker at their Update' 06 (warning, the site is in Spanish) event. Snoop Consulting has a very capable technical team. They are positioning themselves to become a leading company of added-value services for information technologies in the region, focused mainly Marcos
Categories: BI & Warehousing

Time Series Revisited

Marcos Campos - Sat, 2006-10-28 17:55
I have been asked a couple of times for a script that would reproduce the results in the time series forecasting series. I finally managed to do it. In the process I found out that a couple of the queries needed to be tuned:In the airline example described in Part 2, the normalization shift and scale parameters were computed using the whole data. A better methodology would be to use only the Marcos
Categories: BI & Warehousing

Oracle Data Mining 10gR2 Code Generation Release Now Available on OTN

Marcos Campos - Tue, 2006-10-10 19:24
I have just received this from Product Management:We are pleased to announce the newOracle Data Mining 10gR2 Code Generation releaseis now available for download (RTM) on OTN.This new ability to go directly from a data analyst building predictive models to having working in-database PL/SQL code for implementing a complete data mining solution is unrivaled in the industry. There is no data Marcos
Categories: BI & Warehousing


Subscribe to Oracle FAQ aggregator - BI & Warehousing