BI & Warehousing

Announcing PeopleSoft Cloud Manager Support for Oracle Cloud Infrastructure

Tim Dexter - Fri, 2018-05-18 19:45

Oracle released PeopleSoft Cloud Manager in 2017 featuring in-depth automation to help accelerate adoption of Oracle Cloud (Classic) as an efficient deployment platform for PeopleSoft customers. With the excitement generated around Oracle Cloud Infrastructure (OCI)--a cloud designed for the enterprise customer--several customers and partners have been looking forward to taking advantage of the enhanced OCI with PeopleSoft Cloud Manager.  Oracle is pleased to announce Cloud Manager’s support for OCI beginning with today’s release of PeopleSoft Cloud Manager Version 6.

So, what is new and exciting in PeopleSoft Cloud Manager Version 6?  For the first time, there are two images provided: one for OCI, and the other for OCI Classic.  The Cloud Manager Image 6 for OCI supports a number of OCI features, including Regions, Virtual Cloud Networks, Subnets, Compute and DB System platforms.  With this image, instances will be provisioned on VM shapes.  Customers can lift and shift PeopleSoft environments from on-premises to OCI using the same approach they used OCI Classic.

For PeopleSoft Cloud Manager on OCI Classic, we have enabled support for the lift and shift of on-premises databases encrypted with Oracle Transparent Data Encryption (TDE).  TDE offers another level of data security that customers are looking for as data is migrated to the cloud.  A ‘Clone to template’ option is also available for encrypted databases. 

The lift utility requires a few parameters for TDE so that the encrypted database may be packaged and lifted to the cloud.

During shift process, the same parameters are required to deploy the lifted database.

Customers have also requested an enhancement to support non-Unicode databases for PeopleSoft environments.  PeopleSoft Cloud Manager Version 6 supports lift and shift of environments that use non-Unicode Databases.  Unlike image 5, a conversion of the on-premises database to Unicode is no longer required prior to using Cloud Manager’s Lift and Shift automation.

To get your hands on the new Cloud Manager images, go to the Oracle Marketplace and look for either the OCI-Classic image or the OCI image…or try both!   Be sure to review the documentation and additional important information mentioned in the Marketplace listings.

We are excited to combine the automation of provisioning and maintenance that PeopleSoft Cloud Manager provides with the robust benefits of Oracle Cloud Infrastructure.  Combining support for OCI with the additional features of non-Unicode databases and TDE encrypted databases, we expect all customers to benefit from the latest Cloud Manager image, using whichever Oracle Cloud is right for you. 

Stay tuned for additional information and more Cloud Manager features.  Now, off to the next image!

 

Categories: BI & Warehousing

Emerging Tech Helps Progressive Companies Deliver Exceptional CX

Tim Dexter - Fri, 2018-05-18 19:18

It’s no secret that the art of delivering exceptional service to customers—whether they’re consumers or business buyers—is undergoing dramatic change. Customers routinely expect highly personalized experiences across all touchpoints, from marketing and sales to service and support. I call each of these engagements a moment of truth—because leaving customers feeling satisfied and valued at each touchpoint will have a direct bearing on their loyalty and future spending decisions.

This is why customer experience (CX) has become a strategic business imperative for modern companies. Organizations that provide effective, well-integrated CX across the entire customer journey achieved compound annual growth rates of 17%, versus the 3% growth rates logged by their peers who provided less-effective customer experiences, according to Forrester’s 2017 “Customer Experience Index.”

Fortunately, it’s becoming easier to enter the CX winner’s circle. AI, machine learning, IoT, behavioral analytics, and other innovations are helping progressive companies capitalize on internal and third-party data to deliver highly personalized communications, promotional offers, and service engagements.

How can companies fully leverage today’s tools to support exceptional CX? If they haven’t already done so, companies should start evolving away from cloud 1.0 infrastructures, where an amalgam of best-of-breed services runs various business units. These standalone cloud platforms might have initially provided quick on-ramps to modern capabilities, but now, many companies are paying a price for that expediency. Siloed data and workflows hinder the smooth sharing of customer information among departments. This hurts CX when a consumer who just purchased a high-end digital camera at a retail outlet, for example, webchats with that same company’s service department about a problem, and the service team has no idea this is a premium customer.

In contrast, cloud 2.0 is focused on achieving a holistic view of customers—thanks to simplified, well-integrated services that support each phase of the customer journey. Eliminating information silos benefits companies by giving employees all the information they need to provide a tailored experience for every customer.

Achieving modern CX requires the right vendor partnerships. That starts with evaluating cloud services according to how complete, integrated, and extensible the CX platform is for supporting the entire customer journey. One option is the Oracle Customer Experience Cloud (Oracle CX Cloud) suite, an integrated set of applications for the entire customer lifecycle. It’s complemented by native AI capabilities and Oracle Data Cloud, the world’s largest third-party data marketplace of consumer and business information, which manages anonymized information from more than a billion business and 5 billion consumer identifiers. This means that business leaders, besides understanding customers based on their direct interactions, can use Oracle Data Cloud for insights into social, web surfing, and buying habits at third-party sites and retailers and then apply AI to find profitable synergies.

As new disruptive technologies come to the market—whether that’s the mainstreaming of IoT or drones for business—companies will be under constant pressure to integrate these new capabilities to improve their CX strategies. Modern, integrated cloud services designed for CX don’t support just today’s innovations. With the right cloud choices, companies can continually evolve to meet tomorrow’s CX challenges.

(Photo of Des Cahill by Bob Adler, The Verbatim Agency)

Categories: BI & Warehousing

5 Subjects Every Computer Science Student Should Learn

Tim Dexter - Fri, 2018-05-18 18:55

I was fortunate this year to attend the Association for Computer Machinery’s SIGCSE (Special Interest Group on Computer Science Education) conference, where there was a good deal of conversation about what a modern computer science curriculum should include.

Technology changes quickly and it can be difficult for academic programs to keep pace. Still, if computer science students are to contribute meaningfully to the field in either industry or research jobs, it’s critical that they learn modern computing skills. Here are five subjects I think every higher education institution should teach their undergraduate computer science majors:

1. Parallel Programming

The single, standalone server with one CPU has gone the way of the dodo bird, displaced by the cloud, server farms and multithreaded parallel processors. Yet colleges and universities are still mainly teaching their undergraduates sequential programming—programs that execute instructions one after the other—as they have for decades.

Modern computing environments and massive data sets demand not just that we process multiple instructions simultaneously across multiple servers (distributed computing), but also that programs be written to process multiple instructions simultaneously on multicore chips within multiple servers and devices.

Too often, parallel programming is relegated to a single chapter in a textbook, easily skipped when time in the semester runs short. To prepare students for high-performance computing, big data, machine learning, blockchain and more, we must teach them to both think and program in parallel.

2. Green Programming

With the ubiquity of battery-driven computers, energy efficiency is more important than ever. The more we ask our smart devices to do, the more energy they need to do it and the more quickly they exhaust their batteries. The same is true for massive server clusters, where fires related to energy-consumption are not uncommon as we demand faster and faster processing of more and more data.

How you architect a software program directly affects how much energy is needed to execute the program, yet few undergraduate programs teach students about this relationship. In a fast-warming world, one in which we dream big dreams about all the ways artificial intelligence and high-performance computing will make our lives better, it is imperative that we write energy-optimized software. Students will not be able to do that if we don’t teach them how.

3. Collaborative Development

Academia persists in trying to measure what individual students know. In most programming classes, students start from a blank screen and write clean code independently or, less often, with a partner.

But this isn’t how software is engineered in the real world. Professional software engineers almost always start with someone else’s code and work collaboratively in large groups to modify, improve and correct that code, which is then integrated with code written by other engineers in other groups.

It’s common for software development groups to include people from different countries, in different time zones. Working effectively requires team members to communicate well in different languages and across different cultures. It also means that someone else needs to be able to look at your code and know what it does, so following formatting standards and providing clear commenting are critical.

However, in our desire to ensure that each student understands every programming concept and rule of syntax, we overlook opportunities to teach collaborative software development and help students develop critical professional skills.

4. Hardware Architecture

In the minds of most college students, IBM, Intel, and AMD—the inventors and developers of the multicore processor—are old news…old companies founded by old guys. Mobile applications are where the action is.

But mobile apps are driven by data, usually by a lot of data, and they won’t be of much use without the processors, databases and networks that power them.

Computing works and advances based on the entire system, from the power source to the user interface, and students will be more successful if they know how to open the box and “kick the tires.” They can then optimize for energy efficiency and write parallel code that makes use of new hardware architectures. They can manage caching, memory architecture and resource allocation issues. They can explain and explore quantum computing.

Computer science doesn’t stop at software or coding. Students need foundations in hardware architecture, too, including electrical engineering and physics. We need computer scientists who can test and push the boundaries of hardware just as much as they push what can be achieved with software.

5. Computer History and Ethics

Something I heard at the Turing 50th Anniversary celebration last summer has stuck with me: Computing is not neutral. It can be used for good or evil. It can be used to help people and it can be used to manipulate and harm them.

For several decades now, we have been making computing advances for the sake of computing, because what we can make computers do is cool, because the challenge of the next thing is too alluring to pass up, because there is money to be made if we can do “X.”

Just because we can do something with computing, however, doesn’t mean we should. Computing power is so great that we need policies to regulate and manage it, in order to protect and benefit people.

It’s important for students of computing to understand its history and to take courses grounded in ethics so they can make responsible decisions and guide others. They should know computing’s historical villains and heroes, its inventors and detractors, and how it has been used to benefit and hurt people. The old saw applies here: If we do not learn our history, we are doomed to repeat it.

Even in a crowded curriculum, we must ensure students are gaining the skills and knowledge they need to become technology innovators, business leaders and positive contributors to society in the coming decades. This list is only a starting point.

Alison Derbenwick Miller is vice president of Oracle Academy.

Categories: BI & Warehousing

How Blockchain Will Disrupt the Insurance Industry

Tim Dexter - Fri, 2018-05-18 18:49

The insurance industry relies heavily on the notion of trust among transacting parties. For example, when you go to buy car insurance you get asked for things like your zip code, name, age, daily mileage, and make & model of your car. Other than, maybe, the make & model of your car you can pretty much falsify other information about yourself for a better insurance quote. Underwriters trust that you are providing the correct information, which is one of the many risks in the underwriting business.

Enterprise blockchain platforms such as one from Oracle essentially enables trust-as-a-service in such interactions. Participants (insurer and insured) need to come together to do business, but they do not necessarily trust each other. Blockchain provides a scalable mechanism to securely and easily enable trust in such scenarios. There are 4 key properties of Blockchain that enable trust-as-a-service:

  1. Transparency of digital events and transactions it manages,
  2. Immutability of records stored on the blockchain. through append-only time-stamped and hashed records,
  3. Security and assurance that records stored on blockchain aren't compromised through built-in consensus and encryption mechanisms,
  4. Privacy through cryptography

Blockchain can be a good solution for a number of insurance use cases such as:

  • Reducing frauds in underwriting and claims by validating data from customers and suppliers in the value chain
  • Reducing claims by offering tokenized incentives to promote safer driving behavior by capturing data from insured entities like motor vehicles
  • Enabling pay-per-mile billing for insurance by keeping verifiable records of miles traveled
  • And, in the not so distant future, using blockchain to determine liability in case of an accident between two autonomous vehicles by using blockchain to manage timestamped immutable records of decisions made by deep-learning models from both autonomous vehicles right before the accident.

Besides these use cases, blockchain has potential to eliminate intermediaries, improve transparency of records, eliminate manual paperwork, and error-prone processes, which together can deliver orders of magnitude improvement in operational efficiency for businesses. Of course, there are other types of insurance such as healthcare, reinsurance, catastrophic events insurance, property and casualty insurance, which would have some unique flavor of use cases but they would similarly benefit from blockchain to reduce risk and improve business efficiency.

There is no question that blockchain can, potentially, be a disruptive force in the insurance industry. It would have to overcome legal and regulatory barriers before we see mass adoption of blockchain among the industry participants. 

If you are working on an interesting project related to the use of Blockchain for insurance industry feel free to get in touch by leaving a comment or contact us through social media or Oracle sales rep. We’d be glad to help you connect with our subject matter experts and with your industry peers who may be working on similar use cases with Oracle. For more information on Oracle Blockchain, please visit Oracle Blockchain home pages here, and here

 

 

Categories: BI & Warehousing

Oracle's No-Cost Platinum-Level Support Is the New Baseline in the Cloud Market

Tim Dexter - Fri, 2018-05-18 18:36

Companies may give up their servers, storage, and entire data centers when they move to the cloud, but their need for support services doesn’t go away, it changes. Recognizing a growing need for enterprise-class support in the cloud, Oracle is making its Platinum-level support services available at no additional cost to all customers of Oracle Fusion software-as-a-service applications.

“Our objective is to put out a service capability that is simply the best—bar none,” said Oracle CEO Mark Hurd, in announcing that a range of support services would be available for Oracle Fusion enterprise resource planning, enterprise performance management, human capital management, supply chain, manufacturing, and sales and service cloud applications.

The SaaS support services include 24/7 rapid-response technical support, proactive technical monitoring, success planning, end-user adoption guidance, and education resources.

“Most of our customers are going to cloud,” Hurd said in a briefing with journalists at Oracle’s headquarters in Redwood Shores, California. As that happens, he said, “it’s important for someone in the industry, particularly an industry leader in these mission-critical applications, to take a position” on what level of service that transition demands.

“SaaS application support offerings need to become more agile and responsive,” Hurd added. “We need to provide our SaaS customers with everything they need for rapid, low-cost implementations and a successful rollout to their users.”

Catherine Blackmore, Oracle group vice president of North America Customer Success, said Oracle will also offer new advanced services, including dedicated support and certified expertise, for customers that need a higher level of support. “We have a shared interest in our customers’ success, so we’re going above and beyond to ensure our customers have everything they need to succeed,” she said.

Cloud Levels the Playing Field

Oracle also announced the names of first-time cloud customers and others that are expanding their use of Oracle Cloud services. They include Alsea, Broadcom, Exelon, Gonzaga University, Heineken Urban Polo, Providence St. Joseph Health, Sinclair Broadcast Group, and T-Mobile US.

In a Q&A with the journalists, Hurd was asked about his outlook for SaaS adoption outside of the United States and, in particular, in the Latin America region. He said modern cloud applications can be “game changing” for businesses in places where outdated software applications are still the norm.

“You don’t need armies of experts and system integrators,” Hurd said.

Oracle develops thousands of features that are made available regularly to its SaaS application customers. “That’s a feature stream you don’t have to manage from a data center that you don’t have to operate,” Hurd said.

Self-Driving Technology

Hurd pointed to Oracle’s development of autonomous technologies, including the recently introduced Oracle Autonomous Data Warehouse Cloud Service, as another big area of focus at the company. “It gets upgraded, optimized, secured, patched, and tuned, all automatically without any human intervention,” he said.

As the next step in the delivery of autonomous cloud services, Oracle announced the availability of three new Oracle Cloud Platform services with built-in artificial intelligence and machine learning algorithms: Oracle Autonomous Analytics CloudOracle Autonomous Integration Cloud, and Oracle Autonomous Visual Builder Cloud.

Categories: BI & Warehousing

Modern Customer Experience 2018 was Legendary

Tim Dexter - Fri, 2018-05-18 17:53

During his keynote at Modern Customer Experience 2018, Des Cahill, Head CX Evangelist, stated that CX should stand for Continuous Experimentation. He encouraged 4,500 enthusiastic marketers, customer service, sales, and commerce professional us to try new strategies, to take risks, strive to be remarkable, and triumph through sheer determination.

Casey Neistat echoed Des, challenging us to “do what you can’t,” while best-selling author Cheryl Strayed inspired us to look past our fears and be brave. “Courage isn’t success,” she reminded us, “it’s doing what’s hard regardless of the outcome.”

CX professionals today face numerous challenges: the relentless rise of customer expectations, the accelerating pace of innovation, evolving regulations like GDPR, increase ROI, plus the constant pressure to raise the bar. Modern Customer Experience not only inspired attendees to become the heroes of their organization, but it armed each with the tools to do so.

If you missed Carolyne-Matseshe Crawford, VP of Fan Experience at Fanatics talk about how her company’s culture pervades the entire customer experience, or how Magen Hanrahan VP of Product Marketing at Kraft Heinz is obsessed with data driven marketing tactics, give them a watch. And don’t miss Comcast’s Executive VP, Chief Customer Experience Officer, Charlie Herrin, who wants to build proactive customer experience and dialogue into Comcast’s products themselves with artificial intelligence.

The Modern Customer Experience X Room showcased CX innovation, like augmented and virtual reality, artificial intelligence, and the Internet of Things. But it wasn’t all just mock-ups and demos, a Mack Truck, a Yamaha motorcycle, and an Elgin Street Sweeper were on display, showcasing how Oracle customers put innovation to use to create legendary customer experiences.

Attendees were able to let off some steam during morning yoga and group runs. They relived the 90s with Weezer during CX Fest, and our Canine Heroes from xxxxx were a highlight of everyone’s day.

But don’t just take it from us. Here’s what a few of our attendees had to say about the event.

“Modern Customer Experience gives me the ability to learn about new products on the horizon, discuss challenges, connect with other MCX participants, learn best practices and understand we’re not alone in our journey.” – Matt Adams, Sales Cloud Manager, ArcBest

 “Modern Customer Experience really allows me to do my job more effectively. Without it, I don’t know where I would be! It’s the best conference of the year.” – Joshua Parker, Digital Marketing and Automation Manager, Rosetta Stone

We’re still soaking it all in. You can watch all the highlights from Modern Customer Experience keynotes on YouTube, and peruse the event’s photo slideshow. Don’t forget to share your images on social media, with #ModernCX and sign up for alerts when registration for Modern Customer Experience 2019 opens!

Categories: BI & Warehousing

The Most Important Stop on Your Java Journey

Tim Dexter - Fri, 2018-05-18 14:52

Howdy, Pardner. Have you moseyed over to JavaRanch lately? Pull up a stool at the OCJA or OCJP Wall of Fame and tell your tale or peruse the tales of others. 

Ok - I'm not so great at the cowboy talk, but if you're serious about a Java career and haven't visited JavaRanch, you are missing out! 

JavaRanch, a self-proclaimed "friendly place for Java greenhorns [beginners]" was created in 1997 by Kathy Sierra, co-author of at least 5 Java guides for Oracle Press. The ranch was taken over in subsequent years by Paul Wheaten who continues to run this space today.

In addition to a robust collection of discussion forums about all things Java, JavaRanch provides resources to learn and practice Java, book recommendations, and resources to create your first Java program and test your Java skills.

One of our favorite features of JavaRanch remains the Walls of Fame! This is where you can read the personal experiences of other candidates certified on Java. Learn from their processes and their mistakes. Be inspired by their accomplishments. Share your own experience. 

Visit the Oracle Certified Java Associate Wall of Fame

Visit the Oracle Certified Java Professional Well of Fame

Get the latest Java Certification from Oracle

Oracle Certified Associate, Java SE 8 Programmer

Oracle Certified Professional, Java SE 8 Programmer

Oracle Certified Professional, Java SE 8 Programmer (upgrade from Java SE 7)

Oracle Certified Professional, Java SE 8 Programmer (upgrade from Java SE 6 and all prior versions)

Related Content

Test Your Java Knowledge With FREE Sample Questions

Program Your Future With Java

Categories: BI & Warehousing

What's New with Oracle Certification - May

Tim Dexter - Fri, 2018-05-18 14:49
Stay up to date with the Oracle Certification Program.
Keep informed with new exams released into production,
get information on current promotions, and learn about new program announcements. New Exams and Certifications

Oracle Mobile Cloud Enterprise 2018 Associate Developer | 1Z0-927: This certification covers implementation topics of related Oracle Paas Services such as: Visual Builder Cloud Service, Java Cloud Service, Developer Cloud Service, Application Container Cloud Service, and Container Native Apps. This certification validates understanding of the Application Development portfolio and capacity to configure the services.

Oracle Management Cloud 2018 Associate | 1Z0-930: Passing this exams demonstrates the skills and knowledge to architect and implement Oracle Management Cloud. This individual can configure Application Performance Monitoring, Oracle Infrastructure Monitoring, Oracle Log Analytics, Oracle IT Analytics, Oracle Orchestration, Oracle Security Monitoring and Analytics and Oracle Configuration and Compliance.

Oracle Cloud Security 2018 Associate | 1Z0-933: Passing this exam validates understanding of Oracle Cloud Security portfolio and capacity to configure the services. This certification covers topics such as: Identity Security Operations Center Framework, Identity Cloud Service, CASB Cloud Service, Security Monitoring and Analytics Cloud Service, Configuration and Compliance Service, and services Architecture and Deployment.

Oracle Data Integration Platform Cloud 2018 Associate | 1Z0-935: Passing this exam validates understanding of Oracle Application Integration to implement the service. This certification covers topics such as: Oracle Cloud Application Integration basics, Application Integration: Oracle Integration Cloud (OIC), Service-Oriented Architecture Cloud Service (SOACS), Integration API Platform Cloud Service, Internet of Things - Cloud Service (IOTCS), and Oracle's Process Cloud Service.

Oracle Analytics Cloud 2018 Associate | 1Z0-936: Passing this exam provides knowledge required to perform provisioning, build dimensional modelling and create data visualizations. The certified professional can use Advanced Analytics capabilities, create a machine learning model and configure Oracle Analytics Cloud Essbase.

Explore All Certifications

 

How Does the DBA Keep Their Role Relevant? 

By having the skills to meet the new demands for business optimization along with a reputation of continuous learning and improvement. Check out how training + certification keeps a DBA relevant. Read full article.

 

Benefits of Upgrading Your OCA certification to Database 12c Release 2

Building upon the competencies in the Oracle Database 12c OCA certification, the Oracle Certified Professional (OCP) for Oracle Database 12c includes the advanced knowledge and skills required of top-performing database administrators which includes development and deployment of backup, recovery and Cloud computing strategies. Find out how to upgrade with this exam!

Categories: BI & Warehousing

Oracle BI Publisher 12.2.1.4 Now Available !!

Tim Dexter - Fri, 2018-04-27 12:35

Last week Oracle BI Suite 12.2.1.4.0 has been released and that includes Oracle BI Publisher 12.2.1.4.0. 

The links to download the installation files, documentation and release notes are available from BI Publisher OTN home page. You can also download the software from Oracle Software Delivery Cloud.

The new features introduced in Analytics Cloud Suite for both Data Visualization and BI Publisher are now available for on-prem. You can find the  list of new features for Data Visualization here and for BI Publisher here.

Upgrading to Oracle Business Intelligence from 12.2.1.x to 12.2.1.4 is an in-place upgrade performed by Upgrade Assistant. Refer to the upgrade guide for details on upgrade from a previous 12c version.

 

Stay tuned for more updates.

Have a great weekend !

Categories: BI & Warehousing

Adding Native Pivot Charts and Tables to your Excel Reports!!

Tim Dexter - Sun, 2018-04-01 23:59

A report in Excel format is a very common requirement and BI Publisher can generate excel output using RTF, XSL or Excel Template. Excel template is recommended when the requirement is to create pixel perfect column width, to use built in excel functions, to create multi-sheet output, to handle preceding zeroes in data, to maintain data formatting, to manage high number of columns of data, etc.

How about adding native charts and pivot tables in the excel report ? Well, excel templates can handle that too.  

There is no wizard in the Excel Template Builder to create charts or pivot table, but you can certainly include Excel Pivot Charts and Pivot Tables in your report using MS Excel features. Here is a step-by-step guide:

 

Step 1: Create Excel Template to build data for Pivot Chart & Pivot Table

Use Excel Template Builder to create Excel Template

Load a sample XML data. Add data column header names.

Use "Insert Field" option from BI Publisher Ribbon Menu and create data place holders as shown below.

You will see an interim dialog box from the Template Builder that a metadata sheet will be created. Click OK on it.

 

Add looping of data using Insert Repeating Group. Select the For Each entry at the repeating node level

 

Preview the output. This will bring all records in the excel sheet in a separate .xls output file.

 

Step 2: Create Pivot Chart & Pivot Table

You can close the output .xls file and stay in the Excel Template. Now select all the data columns to be used in the Pivot Chart and table. You can click on column headers and select the entire column to be included or you can just select the table with column headers and single row of data placeholders. From Excel Menu Insert, select Pivot Chart & Pivot Table option.

 

In the dialog box "Create PivotTable", you can keep selected the option "Select a table or range" and leave the Table/Range that appears by default based on the selection.

You can choose to create the Pivot Chart and Pivot Table in a new work sheet (recommended).Click OK.

This will add a new Sheet in the Excel file and insert a Pivot Table and Chart place holder, with Pivot Table fields on the right panel

Here you can select the fields for the Pivot table and chart, to be depicted as Axis, Legend and Values. In this example we have included Product Type, Product, LOB and Brand as Axis and Revenue as Values.

Please note that by default the function selected under Values is Count. Therefore, select the drop down next to Count function and choose Value Field Settings, where you can change this to Sum function.

 

 

One more thing to note is the presence of Field Buttons in the chart. You can hide these Field Buttons. With Pivot Chart selected, go to Analyze Menu in the Ribbon style Menu, and under Show/Hide section choose "Hide all Field Buttons".

Finally the template will look like this

 

Step 3: Include dynamic data generated by BI Publisher for Pivot Chart & Pivot Table

Right click on Pivot Chart, select PivotChart Options, select Data tab. Here select the option "Refresh data when opening the file". This will bring the data dynamically into the PIvot Chart and Pivot Table.

 

 

You can run preview of the excel output and you will see the pivot table and chart displaying dynamic data.

You will notice blank data appearing in the Pivot Table and Chart. This is due to the way the looping works against the dynamic data. You can hide this blank data by filtering the blank data from the parent field in the pivot table of the output excel file. In this example, we will remove the blank data from Product field and the complete blank section will be removed without affecting rest of the data. To do this, just hover over Product in the right side pane under Pivot Chart Fields and click on the down arrow. This will open the filter options for Product field. Uncheck the Blank value from filter list. 

 

So, this completes the template design and the final output will look as shown below

You can further include excel functions and formula within these pivot table and charts as necessary for your requirement. You can even change the chart type, style etc. to create the most appropriate visual representation of the data. You can upload the excel template on BI Publisher server and run it against live data. You can include as many sheets with different pivot charts and tables, as required for your report. 

Also note that excel template can be run against any data source type in BI Publisher Data Model. Therefore you can use BI Analysis or even run a BIJDBC SQL query against RPD layer, and bring complex calculations, aggregations as a part of your data. 

 

Hope this was helpful. If you want to check the sample template and data, download it from here.

Have a great day !!

Categories: BI & Warehousing

Happy 20th Anniversary to Applied OLAP!

Tim Tow - Sun, 2018-03-18 13:08

Today, March 18, 2018, is the 20 year anniversary of our incorporation! I has been a long journey since that time; here are some of the highlights:
  • 1998 - We were a one-man shop and wrote a reporting and budgeting application for a customer in New York. I spent about 150 nights that year on the road.
  • 1999 - ActiveOLAP for Essbase was released and we earned the trust of two of our long-term customers. It was during the next couple of years that I traveled to the West Coast about 35 times in one year.
  • 2003 - Portions of our web-service technology was acquired by Hyperion Solutions and we wrote the Hyperion Objects product based on that technology.
  • 2007 - The Dodeca Spreadsheet Management System was released.
  • 2014 - We hired our first resource to focus solely on sales. Prior to that, we marketed our software via 'word of mouth'.
  • 2016 - The Dodeca Excel Add-In for Essbase was released and we acquired the DrillBridge product.
During this time, we have grown the company organically without outside investment. While this strategy meant we had slower growth, the benefit is that it has allowed us to focus solely on the needs of our customers and not the needs of 'investors'. It also meant that we 'ate a lot of beans' in the early days. Those were great lessons in the value of a dollar that we carry with us today in the value of the software we provide to our customers.

Thank you to all of our customers. We feel lucky to work with each and every one of you and we continue to learn from each of you. We pledge to continue working hard to make your companies successful.

Tim Tow
Founder and President
Applied OLAP, Inc
Categories: BI & Warehousing

Rittman Mead at OUG Norway 2018

Rittman Mead Consulting - Mon, 2018-03-05 04:45
Rittman Mead at OUG Norway 2018

This week I am very pleased to represent Rittman Mead by presenting at the Oracle User Group Norway Spring Seminar 2018 delivering two sessions about Oracle Analytics, Kafka, Apache Drill and Data Visualization both on-premises and cloud. The OUGN conference it's unique due to both the really high level of presentations (see related agenda) and the fascinating location being the Color Fantasy Cruiseferry going from Oslo to Kiev and back.

Rittman Mead at OUG Norway 2018

I'll be speaking on Friday 9th at 9:30AM in Auditorium 2 about Visualizing Streams on how the world of Business Analytics has changed in recent years and how to successfully build a Modern Analytical Platform including Apache Kafka, Confluent's recently announced KSQL and Oracle's Data Visualization.

Rittman Mead at OUG Norway 2018

On the same day at 5PM, always in Auditorium 2, I'll be delivering the session OBIEE: Going Down the Rabbit Hole: providing details, built on experience, on how diagnostic tools, non standard configuration and well defined processes can enhance, secure and accelerate any analytical project.

If you’re at the event and you see me in sessions, around the conference or during my talks, I’d be pleased to speak with you about your projects and answer any questions you might have.

Categories: BI & Warehousing

Rittman Mead at OUG Norway 2018

Rittman Mead Consulting - Mon, 2018-03-05 04:45
Rittman Mead at OUG Norway 2018

This week I am very pleased to represent Rittman Mead by presenting at the Oracle User Group Norway Spring Seminar 2018 delivering two sessions about Oracle Analytics, Kafka, Apache Drill and Data Visualization both on-premises and cloud. The OUGN conference it's unique due to both the really high level of presentations (see related agenda) and the fascinating location being the Color Fantasy Cruiseferry going from Oslo to Kiev and back.

Rittman Mead at OUG Norway 2018

I'll be speaking on Friday 9th at 9:30AM in Auditorium 2 about Visualizing Streams on how the world of Business Analytics has changed in recent years and how to successfully build a Modern Analytical Platform including Apache Kafka, Confluent's recently announced KSQL and Oracle's Data Visualization.

Rittman Mead at OUG Norway 2018

On the same day at 5PM, always in Auditorium 2, I'll be delivering the session OBIEE: Going Down the Rabbit Hole: providing details, built on experience, on how diagnostic tools, non standard configuration and well defined processes can enhance, secure and accelerate any analytical project.

If you’re at the event and you see me in sessions, around the conference or during my talks, I’d be pleased to speak with you about your projects and answer any questions you might have.

Categories: BI & Warehousing

Spring into action with our new OBIEE 12c Systems Management & Security On Demand Training course

Rittman Mead Consulting - Mon, 2018-02-19 05:49

Rittman Mead are happy to release a new course to the On Demand Training platform.

The OBIEE 12c Systems Management & Security course is the essential learning tool for any developers or administrators who will be working on the maintenance & optimisation of their OBIEE platform.

Baseline Validation Tool

View lessons and live demos from our experts on the following subjects:

  • What's new in OBIEE 12c
  • Starting & Stopping Services
  • Managing Metadata
  • System Preferences
  • Troubleshooting Issues
  • Caching
  • Usage Tracking
  • Baseline Validation Tool
  • Direct Database Request
  • Write Back
  • LDAP Users & Groups
  • Application Roles
  • Permissions

Get hands on with the practical version of the course which comes with an OBIEE 12c training environment and 9 lab exercises.
System Preferences

Rittman Mead will also be releasing a theory version of the course. This will not include the lab exercises but gives each of the lessons and demos that you'd get as part of the practical course.

Course prices are as follows:

OBIEE 12c Systems Management & Security - PRACTICAL - $499

  • 30 days access to lessons & demos
  • 30 days access to OBIEE 12c training environment for lab exercises
  • 30 days access to Rittman Mead knowledge base for Q&A and lab support

OBIEE 12c Systems Management & Security - THEROY - $299

  • 30 days access to lessons & demos
  • 30 days access to Rittman Mead knowledge base for Q&A

To celebrate the changing of seasons we suggest you Spring into action with OBIEE 12c by receiving a 25% discount on both courses until 31st March 2018 using voucher code:

ODTSPRING18

Access both courses and the rest of our catalog at learn.rittmanmead.com

Categories: BI & Warehousing

Spring into action with our new OBIEE 12c Systems Management & Security On Demand Training course

Rittman Mead Consulting - Mon, 2018-02-19 05:49

Rittman Mead are happy to release a new course to the On Demand Training platform.

The OBIEE 12c Systems Management & Security course is the essential learning tool for any developers or administrators who will be working on the maintenance & optimisation of their OBIEE platform.

Baseline Validation Tool

View lessons and live demos from our experts on the following subjects:

  • What's new in OBIEE 12c
  • Starting & Stopping Services
  • Managing Metadata
  • System Preferences
  • Troubleshooting Issues
  • Caching
  • Usage Tracking
  • Baseline Validation Tool
  • Direct Database Request
  • Write Back
  • LDAP Users & Groups
  • Application Roles
  • Permissions

Get hands on with the practical version of the course which comes with an OBIEE 12c training environment and 9 lab exercises.
System Preferences

Rittman Mead will also be releasing a theory version of the course. This will not include the lab exercises but gives each of the lessons and demos that you'd get as part of the practical course.

Course prices are as follows:

OBIEE 12c Systems Management & Security - PRACTICAL - $499

  • 30 days access to lessons & demos
  • 30 days access to OBIEE 12c training environment for lab exercises
  • 30 days access to Rittman Mead knowledge base for Q&A and lab support

OBIEE 12c Systems Management & Security - THEROY - $299

  • 30 days access to lessons & demos
  • 30 days access to Rittman Mead knowledge base for Q&A

To celebrate the changing of seasons we suggest you Spring into action with OBIEE 12c by receiving a 25% discount on both courses until 31st March 2018 using voucher code:

ODTSPRING18

Access both courses and the rest of our catalog at learn.rittmanmead.com

Categories: BI & Warehousing

Confluent Partnership

Rittman Mead Consulting - Mon, 2018-02-12 09:14

Confluent

Here at Rittman Mead, we are continually broadening the scope and expertise of our services to help our customers keep pace with today's ever-changing technology landscape. One significant change we have seen over the last few years is the increased adoption of data streaming. These solutions can help solve a variety of problems, from real-time data analytics to forming the entire backbone of an organisation's data architecture. We have worked with a number of different technologies that can enable this, however, we often see that Kafka ticks the most boxes.

This is reflected by some of the recent blog posts you will have seen like Tom Underhill hooking up his gaming console to Kafka and Paul Shilling’s piece on collating sailing data. Both these posts try and use day to day or real-world examples to demonstrate some of the concepts behind Kafka.

In conjunction with these, we have been involved in more serious proofs of concepts and project with clients involving Kafka, which no doubt we will write about in time. To help us further our knowledge and also immerse ourselves in the developer community we have decided to become Confluent partners. Confluent was founded by the people who initially developed Kafka at LinkedIn and provides a managed and supported version of Kafka through their platform.

We chose Confluent as we saw them as the driving force behind Kafka, plus the additions they are making to the platform such as the streaming API and KSQL are opening a lot of doors for how streamed data can be used.

We look forward to growing our knowledge and experience in this area and the possibilities that working with both Kafka and Confluent will bring us.

Categories: BI & Warehousing

Confluent Partnership

Rittman Mead Consulting - Mon, 2018-02-12 09:14

Confluent

Here at Rittman Mead, we are continually broadening the scope and expertise of our services to help our customers keep pace with today's ever-changing technology landscape. One significant change we have seen over the last few years is the increased adoption of data streaming. These solutions can help solve a variety of problems, from real-time data analytics to forming the entire backbone of an organisation's data architecture. We have worked with a number of different technologies that can enable this, however, we often see that Kafka ticks the most boxes.

This is reflected by some of the recent blog posts you will have seen like Tom Underhill hooking up his gaming console to Kafka and Paul Shilling’s piece on collating sailing data. Both these posts try and use day to day or real-world examples to demonstrate some of the concepts behind Kafka.

In conjunction with these, we have been involved in more serious proofs of concepts and project with clients involving Kafka, which no doubt we will write about in time. To help us further our knowledge and also immerse ourselves in the developer community we have decided to become Confluent partners. Confluent was founded by the people who initially developed Kafka at LinkedIn and provides a managed and supported version of Kafka through their platform.

We chose Confluent as we saw them as the driving force behind Kafka, plus the additions they are making to the platform such as the streaming API and KSQL are opening a lot of doors for how streamed data can be used.

We look forward to growing our knowledge and experience in this area and the possibilities that working with both Kafka and Confluent will bring us.

Categories: BI & Warehousing

Real-time Sailing Yacht Performance - Getting Started (Part 1)

Rittman Mead Consulting - Fri, 2018-01-19 03:54

In this series of articles, I intend to look at collecting and analysing our yacht’s data. I aim to show how a number of technologies can be used to achieve this and the thought processes around the build and exploration of the data. Ultimately, I want to improve our sailing performance with data, not a new concept for professional teams but well I have a limited amount of hardware and funds, unlike Oracle it seems, time for a bit of DIY!

In this article, I introduce some concepts and terms then I'll start cleaning and exploring the data.

Background

I have owned a Sigma 400 sailing yacht for over twelve years and she is used primarily for cruising but every now and then we do a bit of offshore racing.

In the last few years we have moved from paper charts and a very much manual way of life to electronic charts and IOS apps for navigation.

In 2017 we started to use weather modelling software to predict the most optimal route of a passage taking wind, tide and estimated boat performance (polars) into consideration.

The predicted routes are driven in part by a boat's polars, the original "polars" are a set of theoretical calculations created by the boat’s designer indicating/defining what the boat should do at each wind speed and angle of sailing. Polars give us a plot of the boat's speed given a true wind speed and angle. This in turn informs us of the optimal speed the boat could achieve at any particular angle to wind and wind speed (not taking into consideration helming accuracy, sea state, condition of sails and sail trim - It may be possible for me to derive different polars for different weather conditions). Fundamentally, polars will also give us an indication of the most optimal angle to wind to get to our destination (velocity made good).

The polars we use at the moment are based on a similar boat to the Sigma 400 but are really a best guess. I want our polars to be more accurate. I would also like to start tracking the boats performance real-time and post passage for further analysis.

The purpose of this blog is to use our boats instrument data to create accurate polars for a number of conditions and get a better understanding of our boats performance at each point of sail. I would also see what can be achieved with the AIS data. I intend to use Python to check and decode the data. I will look at a number of tools to store, buffer, visualise and analyse the outputs.

So let’s look at the technology on-board.

Instrumentation Architecture

The instruments are by Raymarine. We have a wind vane, GPS, speed sensor, depth sounder and sea temperature gauge, electronic compass, gyroscope, and rudder angle reader. These are all fed into a central course computer. Some of the instrument displays share and enrich the data calculating such things as apparent wind angles as an example. All the data travels through a proprietary Raymarine messaging system called SeaTalk. To allow Raymarine instruments to interact with other instrumentation there is an NMEA-0183 port. NMEA-0183 is a well-known communication protocol and is fairly well documented so this is the data I need to extract from the system. I currently have an NMEA-0183 cable connecting the Raymarine instruments to an AIS transponder. The AIS transponder includes a Wireless router. The wireless router enables me to connect portable devices to the instrumentation.

The first task is to start looking at the data and get an understanding of what needs to be done before I can start analysing.

Analysing the data

There is a USB connection from the AIS hub however the instructions do warn that this should only be used during installation. I did spool some data from the USB port, it seemed to work OK. I could connect directly to the NMEA-0183 output however that would require me to do some wiring so will look at that if the reliability of the wireless causes issues. The second option was to use the wireless connection. I start by spooling the data to a log file using nc (nc is basically OSX's version of netcat, a TCP and UDP tool).

Spooling the data to a log file

nc  -p 1234 192.168.1.1 2000 > instrument.log

The spooled data gave me a clear indication that there would need to be some sanity checking of the data before it would be useful. The data is split into a number of different message types each consisting of a different structure. I will convert these messages into a JSON format so that the messages are more readable downstream. In the example below the timestamps displayed are attached using awk but my Python script will handle any enrichment as I build out.

The data is comma separated so this makes things easy and there a number of good websites that describe the contents of the messages. Looking at the data using a series of awk commands I clearly identify three main types of messages. GPS, AIS and Integrated instrument messages. Each message ends in a two-digit hex code this can be XOR'd to validate the message.

Looking at an example wind messages

We get two messages related to the wind true and apparent the data is the same because the boat was stationary.

$IIMWV,180.0,R,3.7,N,A*30
$IIMWV,180.0,T,3.8,N,A*30

These are Integrated Instrument Mast Wind Vain (IIMWV) * I have made an assumption about the meaning of M so if you are an expert in these messages feel free to correct me ;-)*

These messages break down to:

  1. $IIMWV II Talker, MWV Sentence
  2. 180.0 Wind Angle 0 - 359
  3. R Relative (T = True)
  4. 3.7 Wind Speed
  5. N Wind Speed Units Knots (N = KPH, M = MPH)
  6. A Status (A= Valid)
  7. *30 Checksums

And in English (ish)

180.0 Degrees Relative wind speed 1.9 Knots.

Example corrupted message

$GPRMC,100851.00,A,5048.73249,N,00005.86148,W,0.01**$GPGGA**,100851.00,5048.73249,N,00005.8614$GPGLL,5048.73249,N,00005.86148,W,100851.0

Looks like the message failed to get a new line. I notice a number of other types of incomplete or corrupted messages so checking them will be an essential part of the build.

Creating a message reader

I don't really want to sit on the boat building code. I need to be doing this while traveling and at home when I get time. So, spooling half an hour of data to a log file gets me started. I can use Python to read from the file and once up and running spool the log file to a local TCP/IP port and read using Python socket library.

Firstly, I read the log file and loop through the messages, each message I check to see if it's valid using the checksum, line length. I used this to log the number of messages in error etc. I have posted the test function, I'm sure there are better ways to write the code but it works.

#DEF Function to test message
 def is_message_valid (orig_line):

  #check if hash is valid
  #set variables
  x = 1
  check = 0
  received_checksum = 0
  line_length = len(orig_line.strip())

  while (x <= line_length):="" current_char="orig_line[x]" #checksum="" is="" always="" two="" chars="" after="" the="" *="" if="" "*":="" received_checksum="orig_line[x+1]" +="" orig_line[x+2]="" #check="" where="" we="" are="" there="" more="" to="" decode="" then="" #have="" take="" into="" account="" new="" line="" line_length=""> (x+3):
        check = 0

      #no need to continue to the end of the 
      #line either error or have checksum
      break

    check = check^ord(current_char)
    x = x + 1; 
  
  if format(check,"2X") == received_checksum:
    #substring the new line for printing
    #print "Processed nmea line >> " + orig_line[:-1] + " Valid message" 
    _Valid = 1
  else:
    #substring the new line for printing
    _Valid = 0
  
  return _Valid

Now for the translation of messages. There are a number of example Python packages in GitHub that translate NMEA messages but I am only currently interested in specific messages, I also want to build appropriate JSON so feel I am better writing this from scratch. Python has JSON libraries so fairly straight forward once the message is defined. I start by looking at the wind and depth messages. I'm not currently seeing any speed messages hopefully because the boat wasn't moving.

def convert_iimwv_json (orig_line):
 #iimwv wind instrumentation

 column_list = orig_line.split(",")
 
 #star separates the checksum from status
 status_check_sum = column_list[5].split("*")
 checksum_value = status_check_sum[1]
 
 json_str = 
 {'message_type' : column_list[0], 
 'wind_angle' : column_list[1], 
 'relative' : column_list[2], 
 'wind_speed' : column_list[3], 
 'wind_speed_units' : column_list[4], 
 'status' : status_check_sum[0], 
 'checksum' : checksum_value[0:2]}
 
 json_dmp = json.dumps(json_str)
 json_obj = json.loads(json_dmp)

 return json_str

I now have a way of checking, reading and converting the message to JSON from a log file. Switching from reading a file to to using the Python socket library I can read the stream directly from a TCP/IP port. Using nc it's possible to simulate the message being sent from the instruments by piping the log file to a port.

Opening port 1234 and listening for terminal input

nc -l 1234

Having spoken to some experts from Digital Yachts it maybe that the missing messages are because Raymarine SeakTalk is not transmitting an NMEA message for speed and a number of other readings. The way I have wired up the NMEA inputs and outputs to the AIS hub may also be causing the doubling up of messages and apparent corruptions. I need more kit! A bi-direction SeaTalk to NMEA converter.

In the next article, I discuss the use of Kafka in the architecture. I want to buffer all my incoming raw messages. If I store all the incoming I can build out the analytics over time i.e as I decode each message type. I will also set about creating a near real time dashboard to display the incoming metrics. The use of Kafka will give me scalability in the model. I'm particularly thinking of Round the Island Race 1,800 boats a good number of these will be transmitting AIS data.


Real-time Sailing Yacht Performance - stepping back a bit (Part 1.1)

Real-time Sailing Yacht Performance - Kafka (Part 2)

Categories: BI & Warehousing

Real-time Sailing Yacht Performance - Getting Started (Part 1)

Rittman Mead Consulting - Fri, 2018-01-19 03:54

In this series of articles, I intend to look at collecting and analysing our yacht’s data. I aim to show how a number of technologies can be used to achieve this and the thought processes around the build and exploration of the data. Ultimately, I want to improve our sailing performance with data, not a new concept for professional teams but well I have a limited amount of hardware and funds, unlike Oracle it seems, time for a bit of DIY!

In this article, I introduce some concepts and terms then I'll start cleaning and exploring the data.

Background

I have owned a Sigma 400 sailing yacht for over twelve years and she is used primarily for cruising but every now and then we do a bit of offshore racing.

In the last few years we have moved from paper charts and a very much manual way of life to electronic charts and IOS apps for navigation.

In 2017 we started to use weather modelling software to predict the most optimal route of a passage taking wind, tide and estimated boat performance (polars) into consideration.

The predicted routes are driven in part by a boat's polars, the original "polars" are a set of theoretical calculations created by the boat’s designer indicating/defining what the boat should do at each wind speed and angle of sailing. Polars give us a plot of the boat's speed given a true wind speed and angle. This in turn informs us of the optimal speed the boat could achieve at any particular angle to wind and wind speed (not taking into consideration helming accuracy, sea state, condition of sails and sail trim - It may be possible for me to derive different polars for different weather conditions). Fundamentally, polars will also give us an indication of the most optimal angle to wind to get to our destination (velocity made good).

The polars we use at the moment are based on a similar boat to the Sigma 400 but are really a best guess. I want our polars to be more accurate. I would also like to start tracking the boats performance real-time and post passage for further analysis.

The purpose of this blog is to use our boats instrument data to create accurate polars for a number of conditions and get a better understanding of our boats performance at each point of sail. I would also see what can be achieved with the AIS data. I intend to use Python to check and decode the data. I will look at a number of tools to store, buffer, visualise and analyse the outputs.

So let’s look at the technology on-board.

Instrumentation Architecture

The instruments are by Raymarine. We have a wind vane, GPS, speed sensor, depth sounder and sea temperature gauge, electronic compass, gyroscope, and rudder angle reader. These are all fed into a central course computer. Some of the instrument displays share and enrich the data calculating such things as apparent wind angles as an example. All the data travels through a proprietary Raymarine messaging system called SeaTalk. To allow Raymarine instruments to interact with other instrumentation there is an NMEA-0183 port. NMEA-0183 is a well-known communication protocol and is fairly well documented so this is the data I need to extract from the system. I currently have an NMEA-0183 cable connecting the Raymarine instruments to an AIS transponder. The AIS transponder includes a Wireless router. The wireless router enables me to connect portable devices to the instrumentation.

The first task is to start looking at the data and get an understanding of what needs to be done before I can start analysing.

Analysing the data

There is a USB connection from the AIS hub however the instructions do warn that this should only be used during installation. I did spool some data from the USB port, it seemed to work OK. I could connect directly to the NMEA-0183 output however that would require me to do some wiring so will look at that if the reliability of the wireless causes issues. The second option was to use the wireless connection. I start by spooling the data to a log file using nc (nc is basically OSX's version of netcat, a TCP and UDP tool).

Spooling the data to a log file

nc  -p 1234 192.168.1.1 2000 > instrument.log

The spooled data gave me a clear indication that there would need to be some sanity checking of the data before it would be useful. The data is split into a number of different message types each consisting of a different structure. I will convert these messages into a JSON format so that the messages are more readable downstream. In the example below the timestamps displayed are attached using awk but my Python script will handle any enrichment as I build out.

The data is comma separated so this makes things easy and there a number of good websites that describe the contents of the messages. Looking at the data using a series of awk commands I clearly identify three main types of messages. GPS, AIS and Integrated instrument messages. Each message ends in a two-digit hex code this can be XOR'd to validate the message.

Looking at an example wind messages

We get two messages related to the wind true and apparent the data is the same because the boat was stationary.

$IIMWV,180.0,R,3.7,N,A*30
$IIMWV,180.0,T,3.8,N,A*30

These are Integrated Instrument Mast Wind Vain (IIMWV) * I have made an assumption about the meaning of M so if you are an expert in these messages feel free to correct me ;-)*

These messages break down to:

  1. $IIMWV II Talker, MWV Sentence
  2. 180.0 Wind Angle 0 - 359
  3. R Relative (T = True)
  4. 3.7 Wind Speed
  5. N Wind Speed Units Knots (N = KPH, M = MPH)
  6. A Status (A= Valid)
  7. *30 Checksums

And in English (ish)

180.0 Degrees Relative wind speed 1.9 Knots.

Example corrupted message

$GPRMC,100851.00,A,5048.73249,N,00005.86148,W,0.01**$GPGGA**,100851.00,5048.73249,N,00005.8614$GPGLL,5048.73249,N,00005.86148,W,100851.0

Looks like the message failed to get a new line. I notice a number of other types of incomplete or corrupted messages so checking them will be an essential part of the build.

Creating a message reader

I don't really want to sit on the boat building code. I need to be doing this while traveling and at home when I get time. So, spooling half an hour of data to a log file gets me started. I can use Python to read from the file and once up and running spool the log file to a local TCP/IP port and read using Python socket library.

Firstly, I read the log file and loop through the messages, each message I check to see if it's valid using the checksum, line length. I used this to log the number of messages in error etc. I have posted the test function, I'm sure there are better ways to write the code but it works.

#DEF Function to test message
 def is_message_valid (orig_line):

  #check if hash is valid
  #set variables
  x = 1
  check = 0
  received_checksum = 0
  line_length = len(orig_line.strip())

  while (x <= line_length):="" current_char="orig_line[x]" #checksum="" is="" always="" two="" chars="" after="" the="" *="" if="" "*":="" received_checksum="orig_line[x+1]" +="" orig_line[x+2]="" #check="" where="" we="" are="" there="" more="" to="" decode="" then="" #have="" take="" into="" account="" new="" line="" line_length=""> (x+3):
        check = 0

      #no need to continue to the end of the 
      #line either error or have checksum
      break

    check = check^ord(current_char)
    x = x + 1; 

  if format(check,"2X") == received_checksum:
    #substring the new line for printing
    #print "Processed nmea line >> " + orig_line[:-1] + " Valid message" 
    _Valid = 1
  else:
    #substring the new line for printing
    _Valid = 0

  return _Valid

Now for the translation of messages. There are a number of example Python packages in GitHub that translate NMEA messages but I am only currently interested in specific messages, I also want to build appropriate JSON so feel I am better writing this from scratch. Python has JSON libraries so fairly straight forward once the message is defined. I start by looking at the wind and depth messages. I'm not currently seeing any speed messages hopefully because the boat wasn't moving.

def convert_iimwv_json (orig_line):
 #iimwv wind instrumentation

 column_list = orig_line.split(",")

 #star separates the checksum from status
 status_check_sum = column_list[5].split("*")
 checksum_value = status_check_sum[1]

 json_str = 
 {'message_type' : column_list[0], 
 'wind_angle' : column_list[1], 
 'relative' : column_list[2], 
 'wind_speed' : column_list[3], 
 'wind_speed_units' : column_list[4], 
 'status' : status_check_sum[0], 
 'checksum' : checksum_value[0:2]}

 json_dmp = json.dumps(json_str)
 json_obj = json.loads(json_dmp)

 return json_str

I now have a way of checking, reading and converting the message to JSON from a log file. Switching from reading a file to to using the Python socket library I can read the stream directly from a TCP/IP port. Using nc it's possible to simulate the message being sent from the instruments by piping the log file to a port.

Opening port 1234 and listening for terminal input

nc -l 1234

Having spoken to some experts from Digital Yachts it maybe that the missing messages are because Raymarine SeakTalk is not transmitting an NMEA message for speed and a number of other readings. The way I have wired up the NMEA inputs and outputs to the AIS hub may also be causing the doubling up of messages and apparent corruptions. I need more kit! A bi-direction SeaTalk to NMEA converter.

In the next article, I discuss the use of Kafka in the architecture. I want to buffer all my incoming raw messages. If I store all the incoming I can build out the analytics over time i.e as I decode each message type. I will also set about creating a near real time dashboard to display the incoming metrics. The use of Kafka will give me scalability in the model. I'm particularly thinking of Round the Island Race 1,800 boats a good number of these will be transmitting AIS data.


Categories: BI & Warehousing

Using MDX for Generated Members in Essbase Reports

Tim Tow - Thu, 2018-01-11 18:30

There are times when Essbase users may need to see an ad-hoc collection of members aggregated together in Essbase, and that isn’t always an easy task.  If it were an aggregation that is needed on a recurring basis, the Essbase administrator may add an alternate hierarchy to assist.  Other times, users might just create a spreadsheet with the desired members in different rows or columns and use Excel formulas to add them together.  In this blog post, I will cover a third option, the use of MDX to create dynamically-generated members, how to run them in Smart View, and how to make them much easier to use in Dodeca.

In order to illustrate how dynamically-generated members can be used, let’s consider an example using the Sample Basic database.  Here is a simple quarterly income statement query that I will use as the basis for this blog post:

SELECT
    {[Year].Children, Year} on COLUMNS,
    Hierarchize(Descendants([Profit]), POST) ON ROWS
FROM 
    Sample.Basic
WHERE 
    ([Market].[New York], [Product].[Colas], Actual)

The results from this simple query look like this:



This MDX is pretty straightforward, but what if you wanted to see how New York and Connecticut would look if they were combined?  This is the question that a generated member can return for you.

Generated members in MDX are created using the WITH MEMBER clause.  Moreover, the generated member can then be used anywhere a normal member can be used, even in a slicer dimension (or what we would call a ‘page field’ in the classic Essbase add-in or a point-of-view in Smart View).  Here is the query modified to use the new generated member:

WITH MEMBER
    [Market].[SelectedMarkets] AS 'SUM({[New York], [Connecticut]})'
SELECT
    {[Year].Children, Year} on COLUMNS,
    Hierarchize(Descendants([Profit]), POST) ON ROWS
FROM
    Sample.Basic
WHERE
    ([Market].[SelectedMarkets], Colas, Actual)

The results from this query look like this:


So far, so good, but there are a couple of things to note.  First, the member displayed in the POV is not a real member; that is to be expected.  This leads to the second thing in that you cannot refresh the query as an ad-hoc analysis; the dynamically generated member name will be replaced with the dimension member name in its place.

To go even further, what if you want to have multiple generated members?  In that case, the syntax is easy as you just continue with another MEMBER clause:

WITH MEMBER
    [Market].[SelectedMarkets] AS 'SUM({[New York], [Connecticut]})'
MEMBER
  [Product].[SelectedProducts] AS 'SUM({[Colas], [Grape]})'
SELECT
    {[Year].Children, Year} on COLUMNS,
    Hierarchize(Descendants([Profit]), POST) ON ROWS
FROM
    Sample.Basic
WHERE
    ([Market].[SelectedMarkets], [Product].[SelectedProducts], Actual)

The results of this query look like this:



The syntax for creating and using generated members is not that difficult, but there are a couple of things that make it a bit more difficult than it should be for end users to use this approach.

First, any time end users start having to deal with scripts of any kind, the level of complexity goes up exponentially.  As one of my mentors used to say, “The difference between zero lines of code and one line of code is much greater than the difference between one line of code and a hundred lines of code”.  In other words, it is hard to get users to deal with code of any kind.

Second, once an end user has to ‘write a line of code’, or script in this case, then they assume the responsibility for it being correct.  As there are differing levels of comfort and skill among users, the risk of error goes up.

Finally, when users use a script like the one used in this example, they have to type in the correct member names or, again, risk error. Here is the new MDX dialog in Smart View 11.1.2.5.720 showing where users type in the MDX including the member names.



To make it much easier for end users, Dodeca does a couple of things.  First, Dodeca developers can configure reports to use MDX without the end user ever having to know that MDX is powering the report ‘under-the-covers’.  Further, Dodeca has flexible Point-of-View selectors that allow the end user to simply pick which members they want to use in the query.

Dodeca report developers use tokens as a sort of substitution variables in the script.  The tokens are replaced in the script at run-time by the members selected by end users.  Here is the same script with tokens in place of the hard-coded values:

WITH MEMBER
  [Market].[SelectedMarkets] AS 'SUM({[T.Market]})'
MEMBER
    [Product].[SelectedProducts] AS 'SUM({[T.Product]})' 
SELECT
    {[Year].Children, Year} on COLUMNS,
    Hierarchize(Descendants([Profit]), POST) ON ROWS
FROM
    Sample.Basic
WHERE
    ([Market].[SelectedMarkets], [Product].[SelectedProducts], Actual)

The Dodeca Essbase Scripts editor has tools to help the report developer create and test MDX scripts.  Here are the Test Tokens available in the editor that allow developers to simulate the values plugged in by the Point-of-View selectors:


And the script itself in the scripts editor which has built-in testing facilities:



Finally, here is a Dodeca view that utilizes the tokenized MDX query and allows users to easily select the members they want dynamically aggregated and the report is produced without the risk of error.



Let me know if you would like to learn more about Dodeca and how it could help your company.


Categories: BI & Warehousing

Pages

Subscribe to Oracle FAQ aggregator - BI &amp; Warehousing