Feed aggregator

Weekly Link Roundup – Jan 20, 2017

Complete IT Professional - Thu, 2017-01-19 18:02
Here’s my weekly link roundup for Jan 20, 2017. What Is This Link Roundup? A “link roundup” is a type of post on a website where the author collates a list of articles they have read recently, and adds them all into a single post. The aim is to make the readers of the site […]
Categories: Development

Ardene Transforms Merchandising to Support International Growth

Oracle Press Releases - Thu, 2017-01-19 14:09
Press Release
Ardene Transforms Merchandising to Support International Growth Fashion Retailer Supports Growth for More Than 400 Locations in the U.S. and Canada

Redwood Shores, Calif.—Jan 19, 2017

Today Oracle announced that leading North America retailer Ardene has implemented Oracle Retail Merchandise Operations Management to support its immediate and long-term international expansion. The fashion retailer is empowering merchants, and planners as they transform their promotions planning, merchandising, and inventory management for its more than 400 stores in Canada and the U.S.

Ardene launched its business transformation when growth began to test the limits of the company’s critical merchandising and store systems. The retailer partnered with systems integrator OLR Group to roll out the Oracle solutions and enable streamlined integration with Ardene’s existing information systems, establishing a connection that leverages the retailer’s legacy sales and customer data. The implementation began with merchandising to establish a foundation for growth.

Oracle will help equip Ardene to better serve customers and improve margins throughout its business. The technology began the implementation by establishing the item master to create a single system of record and execute purchase orders. Ultimately, Ardene will gain a better understanding of actual “landed costs” from global suppliers, to replenish warehouse and store inventory by color and size.   

Ardene leverages Oracle Retail Merchandising solutions to manage inventory and has begun to streamline critical business purchase order processing through a third party. The goal is for the retailer to more effectively plan seasonal buys, optimize vendor relationships, rollout new customer promotions across unique markets, and preserve margins through centralized price management. 

“Oracle empowers retailers to fulfill their business strategy with better insights and processes,” said Ray Carlin, Senior Vice President and General Manager, Oracle Retail. “Ardene is a great example of a retailer that is using Oracle to leverage its tremendous internal resources and talent, while establishing a foundation for innovation and growth.”

Oracle Retail Industry Connect 2017 

Join us at Oracle Industry Connect this spring. The program is designed for and delivered by retailers. On March 20-22, 2017 we will gather as a community in Orlando, FL to share best practices, talk strategy and discover new innovations for retail. Limited to retailers and paying sponsors. Register today: http://www.oracleindustryconnect.com

Contact Info
Matt Torres
About Oracle

Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit www.oracle.com.


Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Safe Harbor

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle Corporation. 

Talk to a Press Contact

Matt Torres

  • 4155951584

Fetch Ref cursor to another ref cursor

Tom Kyte - Thu, 2017-01-19 12:26
Hi, First of all Thank you Gurus! for your time and effort to solve user issues. I have a requirement where I have to consume first row of the big query result set and fetch the rest of the result into a refcursor(for some reason I can't change...
Categories: DBA Blogs

Is it known error with timestamp field update in a compound trigger internal procedure?

Tom Kyte - Thu, 2017-01-19 12:26
I have faced with the error: "ORA-03113: end-of-file on communication channel Process ID: 3942 Session ID: 13 Serial number: 58997" and after that the session is disconnected: "ORA-03114: not connected to ORACLE". In LiveSQL it caused "Intern...
Categories: DBA Blogs

Fetch records in multiple variables

Tom Kyte - Thu, 2017-01-19 12:26
Hi Chris/Connor, I have 3 varaibles in procedure - V_CLIENT_ACCESS_TO_CSP VARCHAR2(100); V_CLIENT_ACCESS_TO_OCA VARCHAR2(100); V_CLIENT_ACCESS_TO_GCMS VARCHAR2(100); I need to fetch the values into these variables using below query - S...
Categories: DBA Blogs

Building a complex logic for grouping the day of the week and time

Tom Kyte - Thu, 2017-01-19 12:26
Hi Tom and Team, Good Afternoon. I have a Location table contains below data in my DB. LOCATION_I LOCATION_SCHED_DAY LOCATION_SCHED_HOUR_FROM LOCATION_SCHED_HOUR_TO NUMERIC_DAY 1528011 MONDAY 9:00 ...
Categories: DBA Blogs

Is there a way to create local hash partition index?

Tom Kyte - Thu, 2017-01-19 12:26
Is there a way to create local hash partition index? I can create global hash partition index, but local does not work.
Categories: DBA Blogs

Is it possible to reuse the same Connection (established from an external JDBC client) within a java stored procedure?

Tom Kyte - Thu, 2017-01-19 12:26
Title: Is it possible to reuse the same Connection (established from an external JDBC client) within a java stored procedure? My application design is as follows Java client (using standard JDBC/Callable statement ) calls > Oracle Function , ...
Categories: DBA Blogs

Difference BW 11G & 12C

Tom Kyte - Thu, 2017-01-19 12:26
Hi, I would like to know the Difference between 11g and 12c, and also Iwanna know which feature makes 12C Cloud and not 11G?
Categories: DBA Blogs


Tom Kyte - Thu, 2017-01-19 12:26
Hi Tom, This is my first here. Really getting fuzzy about this. This is my sql wherein i want to apply the updates on the table <b>ru_receipt_updates</b> to the rows of <b>trans_hdr</b> table located on another server. Though the columns on th...
Categories: DBA Blogs

Pipelined Functions

Tom Kyte - Thu, 2017-01-19 12:26
Hi Team, I am returning a data set to an interface via a ref cursor using the below code:- OPEN pRefCur FOR SELECT ct.ct_value1 AS A, ct.ct_value2 AS B, getArray(ct.ct_value4, ct.ct_value12) AS C, getArray(ct.ct_value5...
Categories: DBA Blogs

Getting ORA-31626: job does not exist and ORA-06512 from my export data pump job

Tom Kyte - Thu, 2017-01-19 12:26
Hello Tom, Please, I need a big help. I have a job scheduled in Enterprise Manager 12c. This job do a full export, but two days ago I started to get the error below: ERROR at line 1: ORA-31626: job does not exist ORA-06512: at "SYS.DBMS_SY...
Categories: DBA Blogs

EBS 12.1 Data Masking Template Certified with EM 13c

Steven Chan - Thu, 2017-01-19 11:56
We're pleased to announce the certification of the E-Business Suite 12.1.3 Data Masking Template for the Data Masking Pack with Enterprise Manager Cloud Control 13c. You can use the Oracle Data Masking Pack with Oracle Enterprise Manager Cloud Control 13c to mask sensitive data in cloned Oracle E-Business Suite environments.  Due to data dependencies, scrambling E-Business Suite data is not a trivial task.  The data needs to be scrubbed in such a way that allows the application to continue to function. 

You may scramble data in E-Business Suite 12.1.3 cloned environments with EM13c using the My Oracle Support Note and template: 

What does masking do in an Oracle E-Business Suite environment?

Based upon the knowledge of the Oracle E-Business Suite architecture and sensitive columns, application data masking does the following:

  • De-identifies the data:  Scramble identifiers of individuals, also known as personally identifiable information (PII).  Examples include information such as name, account, address, location, and driver's license number.
  • Masks sensitive data:  Mask data that, if associated with personally identifiable information (PII), would cause privacy concerns.  Examples include compensation, health and employment information.  
  • Maintains data validity:  Provide a fully functional application.

Related Articles

Categories: APPS Blogs

Oracle E-Business Suite: 250 Security Vulnerabilities Fixed in the Last Year

Oracle has fixed 250 security vulnerabilities in the Oracle E-Business Suite from January 2016 to January 2017.  The past five Oracle Critical Update Updates (CPU) have included double or triple digit number of fixes for Oracle E-Business Suite.  Almost all these security vulnerabilities are exploitable in all versions of Oracle E-Business Suite including 11i, 12.0, 12.1, and 12.2.  Many of the 250 security vulnerabilities fixed are high risk vulnerabilities such as SQL injection, cross-site scripting (XSS), XML external entity attacks, and privilege escalation.

Unless your organization is applying the CPU patches immediately and have hardened the application, the Oracle E-Business Suite is extremely vulnerable and easily exploitable.  Significant defensive measures are required to protect Oracle E-Business Suite especially those with Internet facing modules such as iSupplier, iStore, iRecruitment, and iSupport.   A key layer of defense is Integrigy’s web application firewall for Oracle E-Business Suite, AppDefend, which provides virtual patching for these security bugs and additional protection from generic web application attack like SQL injection and cross-site scripting (XSS) and common Oracle E-Business Suite security misconfigurations.

Oracle E-Business Suite, Oracle Critical Patch Updates
Categories: APPS Blogs, Security Blogs

Oracle E-Business Suite 11i - Critical Patch Updates Extended for Tier 1 Support

As of December 2016, Oracle has extended Critical Patch Update (CPU) support for Oracle E-Business Suite 11.5.10 until October 2017 for additional fee Tier 1 support/Advanced Contract Support (ACS) customers.  Starting with the April 2016 Critical Patch Update (CPU), Oracle E-Business Suite 11.5.10 CPU patches are only available for customers with Tier 1/ACS support contracts.  See My Oracle Support Note ID 1596629.1 for more information.

Almost all security vulnerabilities discovered and patched in Oracle E-Business Suite 12.x are also present and exploitable in 11i.  A significant number of these security bugs are SQL injection bugs allow an attacker to execute SQL as the Oracle E-Business Suite APPS database account.  These attacks can easily compromise the entire application and database.  In the past year, Oracle has fixed 250 security vulnerabilities in Oracle E-Business Suite 11i and R12.

Oracle E-Business Suite 11i customers without Tier 1 support, as well as 12.0 customers, should take immediate take immediate defensive steps to protect the Oracle E-Business Suite 11i, especially those with Internet facing modules such as iSupplier, iStore, iRecruitment, and iSupport.  A key layer of defense is Integrigy’s web application firewall for Oracle E-Business Suite, AppDefend, which provides virtual patching for these security bugs and additional protection from generic web application attack like SQL injection and cross-site scripting (XSS) and common Oracle E-Business Suite security misconfigurations.

Oracle E-Business Suite, Oracle Critical Patch Updates
Categories: APPS Blogs, Security Blogs

vagrant up – get your Oracle infrastructure up an running

Yann Neuhaus - Thu, 2017-01-19 10:55

By using Vagrant to manage your Virtual Machines and Ansible for configuration management and provisioning
you can easily automate the setup of your whole test environment in a standardized way.

If you have never heard about Ansible and Vagrant I try to give you an idea with the following very short  summary. There is a lot of good information available about Ansible and Vagrant.
Please check the provided links at the end of this blog for further information.

What is Vagrant ?

Vagrant is an open source tool on top of some virtualization solution like Oracle VirtualBox. It can automate the creation of VM’s. Additionally vagrant supports provisioning with scripts or with tools like Ansible, Puppet or Chef.
You can download a lot of useful  boxes from here: https://atlas.hashicorp.com/boxes/search

 What is Ansible ?

Ansible is an open source automation platform.
It is a radically simple IT automation engine designed for multi-tier deployments. [https://www.ansible.com/how-ansible-works]

Ansible just uses ssh and does not require agents or other software installed on the target nodes. You simply put your steps into an Ansible playbook, which is an easy to read text-file written in YAML syntax. Your playbook will simply look like documented steps.
Ansible will run the listed tasks described in the playbook on the target servers by invoking Ansible Modules.

Here is a simple example task from a playbook which will add a directory on a server. It uses the Ansible module “file”
- name: add home directory
path: /home/myuser
state: directory

Ansible is quite well known to build up whole test environments including databases like mysql which are easy to install with simple tar balls or rpm files.

Unfortunately in the community of Oracle DBA’s usually Ansible is not on the radar despite there are already good Ansible playbooks available which proofed that you can also use Ansible to provision your whole Oracle Test Environment even with Oracle Real Application Cluster:


Starting from these examples and adapting them for your needs you will experience how quick you will be able to automate your Oracle installations. This is what I did an want to show you here. Please keep in mind that this example is optimized for a fast installation and should not be used as it is for a productive system.

What you’ll get
In this blog I give you an example how to build an Oracle infrastructure from scratch containing
two virtual servers, installed and configured with CentOS 7.2 ,
each hosting an Oracle DB (

  • Step ONE – What you need to prepare once to run this example
      1) the Ansible Playbook and Vagrant configuration for this example
      you can download everything from the git repository. All files are simple text files.
      2) the Oracle binaries
      the Oracle binaries are not included in the download. You have to provide them.
      Please copy the Oracle software zip files into the directory oracle-db-12c-vagrant-ansible/

      3) your Linux host or laptop
      with Network Connection,Oracle VirtualBox , Vagrant and Ansible installed.
      This can be done with your linux package manager.
      You will need Ansible version or higher for this example!
      Please check http://docs.ansible.com/ansible/intro_installation.html for installation details.
      sudo yum install ansible
      You can find the Oracle VirtualBox Download and Installation Guide here:
      Download Vagrant with version 1.8.5 or higher from
      Also install the vagrant hostmanager plugin:
      $ vagrant plugin install vagrant-hostmanager
  • Step TWO – Run it
      Now you are ready to start the whole setup which will create two virtual servers and oracle databases.
      On my laptop with SSD disks and 16 GB RAM this takes about 20 minutes.
      To run this example you will need minimal 8 GB RAM and 10G free disk space
      Go to the directory where you have downloaded this example. Everything will be started from here.
      cd oracle-db-12c-vagrant-ansible
      vagrant up
  • Of cause you do not want to start this without knowing what is going on.
    I will go a little bit into details therefore next week ….

    Further information about Ansible:
    There will be some Introduction Webinars for Ansible coming soon

    you can find more examples at:
    If you want to read a book I can recommend this:
    Ansible: Up and Running
    Print ISBN: 978-1-4919-1532-5
    Ebook ISBN: 978-1-4919-1529-5



    Cet article vagrant up – get your Oracle infrastructure up an running est apparu en premier sur Blog dbi services.

    Details about this incident are available via debug id....

    Joel Kallman - Thu, 2017-01-19 08:07

    In Oracle Application Express, unexpected exceptions can occasionally occur.  Instead of communicating the technical details to an end user who is running the application, an error message will be issued of the form:

    Contact your administrator. Details about this incident are available via debug id "NN".

    But what actually happened?  Who is "your administrator"?  And what exactly are you supposed to do with this information?

    What happened?An internal exception occurred - it either wasn't properly handled by the application's error handler (or there was no handler in place), or an unforeseen exception occurred in the Application Express engine itself, which wasn't properly caught by the error handler of APEX.  If you're logged into the development environment as a developer and running the application, you should be able to see the full error stack and more details of what actually occurred.  But if you're a mere end-user of the application, you are shielded from those details - not only would the details be mystifying to an end user, but it could also be viewed as a security risk, to show the details.

    Who is "your administrator"?
    • A developer of this application, who has access to SQL Commands in the APEX workspace?
    • A developer who can connect directly to the database where this error occurred, connecting as the database user associated with the workspace (commonly referred as the "workspace schema").
    • A database user who has been granted the APEX_ADMINISTRATOR_ROLE (or, beginning in APEX 5.1, also the APEX_ADMINISTRATOR_READ_ROLE).
    • Someone who can connect as SYS or SYSTEM to your database.

    What can you do with this information?The full details of the incident and the error stack at the time of the exception are written to the Debug Messages tables in the Application Express schema.  To view these details, you simply need to query the view APEX_DEBUG_MESSAGES:

    select *
    from apex_debug_messages
    where page_view_id = NN
    order by message_timestamp asc;

    This information is also accessible from the development environment of Application Express.
    • A developer can select their application in App Builder, and then navigate to Utilities -> Debug Messages.
    • A workspace administrator can navigate to Workspace Administration -> Monitor Activity -> Page Views By View.  You can then filter the output by the Debug ID column.

    Oracle Marketing Cloud Teams with Eyeota to Enhance Global Data Offering

    Oracle Press Releases - Thu, 2017-01-19 07:00
    Press Release
    Oracle Marketing Cloud Teams with Eyeota to Enhance Global Data Offering Eyeota data helps Oracle Data Management Platform customers improve ROI and strengthen customer experience across global campaigns

    Redwood Shores, Calif.—Jan 19, 2017

    Oracle Marketing Cloud today announced it is teaming with Eyeota to enable marketers and advertisers to use Eyeota data in the Oracle Data Management Platform to more intelligently target and personalize non-US campaigns to Eyeota’s three billion unique profiles in Europe, APAC, and the Americas. Eyeota data will be integrated into the Oracle Data Management Platform through the Oracle Data Cloud, and Oracle Data Management Platform users can build audiences directly from Eyeota data or by seamlessly combining it with additional data segments from Oracle’s BlueKai Marketplace.

    Data has transformed marketing by giving marketers and advertisers the ability to enhance the customer experience and increase ROI. To extend the benefits of data-driven marketing strategies to international markets, advertisers need to be able to execute targeted, data-driven campaigns in all the markets they operate in, but this can often be extremely complex due to highly fragmented and linguistically diverse markets. The relationship between Oracle Marketing Cloud and Eyeota addresses this challenge by providing a new natively integrated data set that enables Eyeota data in areas such as socioeconomics, interest, intent and B2B to be activated within the Oracle Data Management Platform.

    “To help global and international brands improve key campaign and ROI metrics, Oracle is committed to working with the best data providers in strategic markets across the globe,” said Karen Kokiko, senior product marketing manager, Oracle Marketing Cloud. “Our data management platform is differentiated on features and capabilities, but it truly stands out in terms of the data it has natively integrated into the platform. Eyeota’s audiences reflect the company’s depth of data in the EMEA and APAC regions and this integration dramatically improves our international data offering, making the Oracle Data Management Platform an even more compelling platform.” 

    Eyeota is a leading provider of high-quality audience data that enables brands to more efficiently identify and engage customers by using more than three billion unique profiles to create audience and message consistency across marketing channels. With the new integration, Oracle Data Management Platform users can increase speed to market by selecting, purchasing, and managing Eyeota data from within the Oracle Data Management Platform UI.

    “In today’s market, brands understand the importance of knowing their audiences on an almost intimate level to help them deliver more personalized and relevant content, while enhancing the online experience,” said Kevin Tan, CEO, Eyeota. “Eyeota’s extensive global data segments provide a granular view of audiences to help them improve their outreach strategies. Our integration with the Oracle Data Management Platform offers brands deep insights to better understand their customers, improve audience engagement, and drive significant ROI.”

    The Oracle Data Management Platform is integrated with Oracle Responsys, Oracle Eloqua, and Oracle Social Relationship Management platforms to allow marketers to achieve immediate cross-channel benefits. The Oracle Data Management Platform also enables marketers to seamlessly integrate Eyeota data with additional data from Oracle’s BlueKai Marketplace, which includes more than 30,000 data attributes on two billion consumer profiles drawn from 1,500 data partners.

    The Oracle Marketing Cloud is part of the Oracle Customer Experience (CX) Cloud, one of the industry’s most complete CX solutions. Oracle CX Cloud empowers organizations to improve experiences, enhance loyalty, differentiate their brands, and drive measurable results by creating consistent, connected, and personalized brand experiences across all channels and devices.

    Contact Info
    Simon Jones
    PR for Oracle
    About Eyeota

    Eyeota is the global leader for audience data with 3 billion unique profiles in Europe, APAC and the Americas. The company provides marketers with the data they need to reach the right online audiences and cut campaign waste while also enabling publishers to monetize their audiences more widely. Eyeota is a certified member of the Network Advertising Initiative (NAI), the leading self-regulatory industry association dedicated to responsible data collection and its use for digital advertising, and has received consumer privacy certifications around the world. Eyeota data delivers deep audience insights to help brands understand their customers in a new way - as humans. For more information, visit http://www.eyeota.com/.

    About Oracle

    Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit www.oracle.com.


    Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

    Safe Harbor

    The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle Corporation. 

    Talk to a Press Contact

    Simon Jones

    • +1.415.856.5155

    Oracle Buys Apiary

    Oracle Press Releases - Thu, 2017-01-19 07:00
    Press Release
    Oracle Buys Apiary Creates the Most Comprehensive API Integration Cloud by Adding the Leading API Design and Governance Solution

    Redwood Shores, Calif.—Jan 19, 2017

    Oracle today announced that it has signed an agreement to acquire Apiary, whose pioneering APIFlow solution provides the framework and tools for developing application programming interfaces (APIs) that share enterprise services and data and help create modern, cloud-based applications and experiences.

    Apiary has helped companies create hundreds of thousands of APIs and products that their customers and partners love to use. APIFlow spans the API creation lifecycle, including design, governance, testing, and documentation, while supporting API Blueprint and OpenAPI industry standards. Together, Oracle and Apiary will help companies thrive in the digital economy by comprehensively managing connectivity complexity and API proliferation.

    “Oracle’s API Integration Cloud enables companies to secure, consume, monetize, and analyze APIs,” said Amit Zavery, Senior Vice President, Integration Cloud, Oracle. “With Apiary, Oracle will also provide customers advanced capabilities to design and govern API’s, allowing companies to manage the entire API lifecycle and deliver integrated applications.”

    “Oracle customers will have unique access to a comprehensive API management platform providing control and increased agility, enabling them to focus on innovation,” said Jakub Nešetřil, Founder and CEO, Apiary. “We are excited to join Oracle and bring even more value to customers as part of Oracle’s Integration Cloud.”

    More information about this announcement is available at www.oracle.com/apiary

    Contact Info
    Deborah Hellinger
    Oracle Corporate Communications
    Ken Bond
    Oracle Investor Relations
    About Oracle

    Oracle offers a comprehensive and fully integrated stack of cloud applications and platform services. For more information about Oracle (NYSE:ORCL), visit www.oracle.com.


    Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

    Oracle is currently reviewing the existing Apiary product roadmap and will be providing guidance to customers in accordance with Oracle’s standard product communication policies. Any resulting features and timing of release of such features as determined by Oracle’s review of Apiary’s product roadmap are at the sole discretion of Oracle. All product roadmap information, whether communicated by Apiary or by Oracle, does not represent a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. It is intended for information purposes only, and may not be incorporated into any contract.

    Cautionary Statement Regarding Forward-Looking Statements
    This document contains certain forward-looking statements about Oracle and Apiary, including statements that involve risks and uncertainties concerning Oracle’s proposed acquisition of Apiary, anticipated customer benefits and general business outlook. When used in this document, the words “anticipates”, “can”, “will”, “look forward to”, “expected” and similar expressions and any other statements that are not historical facts are intended to identify those assertions as forward-looking statements. Any such statement may be influenced by a variety of factors, many of which are beyond the control of Oracle or Apiary, that could cause actual outcomes and results to be materially different from those projected, described, expressed or implied in this document due to a number of risks and uncertainties. Potential risks and uncertainties include, among others, the possibility that the transaction will not close or that the closing may be delayed, the anticipated synergies of the combined companies may not be achieved after closing, the combined operations may not be successfully integrated in a timely manner, if at all, general economic conditions in regions in which either company does business may deteriorate and/or Oracle or Apiary may be adversely affected by other economic, business, and/or competitive factors. Accordingly, no assurances can be given that any of the events anticipated by the forward-looking statements will transpire or occur, or if any of them do so, what impact they will have on the results of operations or financial condition of Oracle or Apiary. You are cautioned to not place undue reliance on forward-looking statements, which speak only as of the date of this document. Neither Oracle nor Apiary is under any duty to update any of the information in this document.

    Talk to a Press Contact

    Deborah Hellinger

    • +1.212.508.7935

    Ken Bond

    • +1.650.607.0349

    Time Series Visualisations: Kibana or Grafana?

    Rittman Mead Consulting - Thu, 2017-01-19 05:49

    Grafana has rapidly become one of the de-facto “DevOps” tools for real time monitoring dashboards of time series metrics. In addition to its powerful visualisations, Grafana is not tied to a particular stack or vendor, and supports multiple backend data sources including InfluxDB, Graphite, Elasticsearch and many others which can be added via plugins.

    Another similar tool, Kibana is the data visualisation front end for the Elastic Stack, complementing the rest of the stack which includes Beats, Logstash (ingest) and Elasticsearch itself (storage). With the version 5.x release of the Elastic Stack, Kibana now includes Timelion for interactive time series charts.

    Here at Rittman Mead we are big fans of both tools, and have written about them over the years (see 1, 2, 3). Our industry-leading Performance Analytics solution for OBIEE is built on top of these tools, and takes advantage of the time series features to provide interactive web-based dashboards presenting a “full stack" view of the important metrics relating to OBIEE's performance.

    To give you an idea of what we’ve built, here is a sample dashboard from our Performance Analytics tool. We use both Grafana and Kibana, to present different views of data. The dense dashboards of time series metrics work brilliantly in Grafana:

    To enable the user to view and analyse performance data across multiple dimensions we use Kibana, which does a stirling job:

    With the recent release of Timelion - a time series visualisation plugin for Kibana - out of beta and into the big time, we wanted to ensure we were still using the right tool for the right job. Did we still need Grafana in our stack for visualisation of time series metrics, or could Timelion fill that gap now, and enable us to streamline our platform’s toolset?

    In this article we’ll see how Timelion and Grafana stack up against each other. The intention is not to define which is “best” (a pointless exercise), nor create an unintelligible grid of down-in-the-weeds features that each may or may not support, but to see how the two tools compared in real-world usage, side by side. Which makes it easier to build charts? Which produces a nicer-looking dashboard at the end of it? Which has the best UI and UX for the end user reading and analysing the data? What limitations -if any- are there on data sources and functionality in analysing that data? And ultimately, can we unify our product’s front end on a single one of these tools?

    Introduction to Timelion

    Since version 5 of Kibana, Timelion (pronounced "Timeline") has been included as part of the default installation. Charts are defined using a bespoke query language, which specifies both the source of the data, functions to apply to it, and how it is presented. The query is specified in a textbox in the Timelion interface. In this simple chart here we’re using the expression .es(*) to show the total number of documents in Elasticsearch, over time:

    Every Timelion expression starts with a data source function and continues with a chain of functions that are connected with a dot. Over 20 functions are provided, across three groups:

    1. Data sources - the default is Elasticsearch, and other APIs such as World Bank and Quandl are also available.
      For example in the graph above, the default expression .es(*) (similar to .elasticsearch(*)) shows a count of all documents in Elasticsearch. You can specify details of the Elasticsearch index, mappings and metrics here too, as well as filters.

    2. Data manipulations ranging from simple arithmetic to moving averages, cumulative sums and derivatives
      For example, adding a moving average to the data is as simple as including the function to the end of the expression: .es(*).movingaverage(12)

    3. Themes and styles of the visual elements including bar/point/lines, labels, title and legends. The graph below shows the number of running queries by time extracted from the active session history data in the Oracle database. .es(index=ash*).lines(1,fill=1).title('Running Queries').legend(none).label(false)

    With regards to the available documentation and guides for the developers, the main documentation for Timelion is somewhat sparse. For details of each function you can refer to the documentation on github. Compared to the rest of the excellent Elastic documentation, this is surprising and hopefully now that Timelion is part of the core product its documentation will be brought up to parity - full explanations of features and functions along with examples of usage.

    On the positive side, the query builder text box supports auto-complete of functions and their arguments, and the Timelion interface provides online help too. A downside to this minimalist Timelion page is the size of the expression textbox. As you will read more in this post, it wouldn’t take long before you need to add more than one metric and a few styles to a visualisation which means having too many words in the textbox that can’t be seen, scrolled and edited easily:

    If you are a beginner, to avoid the confusion over typos and errors, try building the expressions step by step and add functions gradually. The blog here nicely explains how to gradually create Timelion expressions.

    Of special note in the data manipulation functions that Timelion provides are the statistical analysis ones:

    • .trend() : add a trendline using a specified regression algorithm to your graph
    • .holt(): an early version of this function, which samples the beginning of a series and use it to forecast what should happen via several optional parameters.

    These are useful for our performance monitoring dashboards, enabling us to show things such as the point at which you would run out of memory/disk space if you continued to consume resources at your current rate.

    Related to this concept is Prelert, which Elastic acquired next year and is expected to be part of a future X-Pack release. Whilst dashboard-based analysis is useful, once a clear pattern on which we want to alert is identified we can bring in Watcher to provide real time notifications to pager systems etc.

    Introduction to Grafana

    Grafana is an open source feature rich dashboard and graph editor that is rapidly becoming accepted as one of the best time-series metric visualisation tools available. Grafana has gained its popularity thanks to its simplicity, ease of use and snazzy look and feel that attracts many users. You can read more about Grafana in an earlier article that we wrote on the Rittman Mead blog here. Here is the kind of dashboard you can easily build with Grafana:

    Most of the configurations in Grafana are done via a comprehensive graph editor interface:

    In the Grafana editor queries are generally built entirely through the GUI. Manually specified queries are used in cases such as accessing advanced functionality, and for specifying Lucene queries for in order to access data held in Elasticsearch. In terms of support for Elasticsearch, the latest version of Grafana at the time of writing this post (v4.1.1) supports both Elasticsearch v2 and v5. From my time spent working with Grafana 4.1.1 and Elasticsearch v5 I haven’t found it to be as stable as the long-standing data sources such as InfluxDB and Graphite (or even Elasticsearch v2). As an example, if a chart is configured incorrectly (for example settings for null values), Grafana is not as intuitive in returning no results or throw a descriptive error explaining the issue; instead the graph seems locked and the only possible solution for this behaviour seems to be deleting the chart and recreating it from scratch.

    As well as data sources the graph editor includes settings covering display styles such as titles, templates, axis and legends.

    A interesting new addition to the Grafana family is the alerting engine which allows users to attach rules to the dashboard panels. Once dashboards are saved Grafana will extract the alert rules into a separate alert rule storage and schedule them for evaluation.

    Side-by-Side : Presenting the Data

    On the face of it, the output from Grafana and Timelion can be remarkably similar:

    Grafana on the right and Timelion is the left graph

    However, there are a few differences between the two tools that are worth digging into here. They are mainly on the display configuration part and simplicity of the user experience.

    As mentioned, Grafana’s chart editor has a clear interface over the multitude of options available for refining the presentation of the data.

    Timelion also supports chart formatting, but with fewer options than Grafana. It also depends on the user concatenating the correct functions onto the data query expression as we saw above. For example to add a graph that has a “Running Queries” title, a legend on the top right of the plot, not labeled axes and data shown with a 1px width line, you would need to hand-code the this expression: .lines(1,fill=1).title('Running Queries').legend(ne).label(false)

    Grafana offers significantly greater flexibility in the formatting of the chart. One example is displaying metrics of different units such as time, currency and data. Grafana can automatically scale axes based on the units (bytes -> MB -> GB). The following Grafana graph shows disk usage from our monitored application stored in Elasticsearch. The disk usage metric on the Y axis is in Kilobytes, which Grafana has automagically scaled to the appropriate magnitude (MiB) in the labelling:

    The same could be done manually in Timelion by specifying the appropriate conversion, but this is a hardcoded option compared to Grafana’s dynamic one, and even then wouldn’t have the varying labeling that Grafana does above (KiB initially, switching to MiB subsequently)

    Grafana also supports the rendering of negative values on the Y axis, which is just not possible in Timelion. As well as genuinely negative data values (for example, temperature recordings below zero degrees), using transform feature of Grafana it is possible to invert particular series so as to aid the comprehension of the data as seen here:

    Another nice feature that Grafana has - and unfortunately Timelion doesn’t - is the ability to show metric values in the legend itself. It’s a great way to see key values at a glance, without requiring a separate table or the user to hover over the data points.

    Side-by-Side : Interacting with the Data

    Grafana and Kibana are also different in terms of the level and ease with which it is possible to interact with the charted data. Both Kibana and Grafana support the drag-select of time periods on a chart to zoom into detail, with the rest of the charts on the same dashboard updating to show the same time period too. However, Kibana is much more feature-rich in this area. As a front end to Elasticsearch it supports ad-hoc text search of your data. It also allows users to automatically drill down into data, by clicking on a value in a chart to show details just for that. In the OBIEE monitoring dashboard below (built in Kibana), Active Session History data is filtered for the session_states in “Waiting” and “On CPU” - this filter was created by the user simply by clicking on the data points in one of the charts, and can be toggled dynamically from the same interface.

    This interactivity is supported by Timelion too. The es() datasource function includes an argument called “kibana”. This argument defines whether the visualisation should follow the filters applied to the rest of the Kibana dashboard or not, for example:

    Whilst it is possible to specify Elasticsearch Lucene queries in Grafana and use term filters in the editor, these are local to the graph. With some use of variables it can be possible to enable a degree of global filtering on a single Grafana dashboard but this is a bespoke solution per-dashboard, rather than the out-of-the-box functionality that Kibana provides.

    Grafana does enable you to toggle the display of data in a chart, by clicking on the measure label in the legend, seen above.


    Comparing Kibana and Timelion to Grafana, it is true that they do a similar job displaying time series metrics - with pros and cons on each side.

    Grafana’s graph editor offers an amazing interface with regards to the options available for refining the presentation of the data. Grafana is not only an straightforward development tool but also adds a huge amount of value to the resulting dashboards making them easier to read and analyse by the end users

    On other hand, Timelion is just one of many visualisations that Kibana provides (including Tile Map and Tag Cloud), meaning that dashboards can be built which are less dense with numbers and time series but information is shown through variety of visualisations. Unfortunately Timelion and its expression editor at its current version seem slightly immature and relatively limited. A few more additional display options plus a nicer editor would put Timelion in a better position in comparison.

    So, for now, we’ll be sticking with our dual approach of both Grafana and Kibana. Grafana provides our pure time-series metric dashboards, with the ease-of-building being one of the key factors, along with the rich formatting capabilities and its support for a data sources rather than Elasticsearch. Kibana does an unbeatable job of dashboards enabling rich exploration of metrics across dimensions, rendered in a greater number of possible visualisation forms. Timelion is a great first step, but ultimately just can’t compete with Grafana.

    This is a fast-moving area of tool development, and you can bet that Grafana and Kibana are going to continue developing at a rate of knots - which as users and developers is great news!

    Categories: BI & Warehousing


    Subscribe to Oracle FAQ aggregator