Feed aggregator

Flipkart, ecommerce, machine learning, and free advice

Abhinav Agarwal - Sat, 2015-07-04 13:08
I wrote about the obsession of Flipkart (and Myntra) with "mobile-only" without even having an iPad-optimized app! I also talked about the stunning advances being made in voice-search by using machine learning, cognitive learning, natural language processing, even as voice-based search capabilities of e-commerce companies - including Amazon - remain abysmal. Finally, I also included several use-cases that these companies need to work on incorporating into their capabilities.

That piece, Flipkart, Focus and Free Advice, appeared in DNA on June 27th, 2015.


My earlier pieces on the same topic:
  1. Flipkart vs Amazon: Beware the Whispering Death - 20th April '15 (blog, dna)
  2. Mobile Apps: There’s Something (Profitable) About Your Privacy - 18th April '15  (blog, dna)
  3. Mobile advertising and how the numbers game can be misleading - 14th April '15  (blog, dna)
  4. Is Flipkart losing focus - 12th April '15  (blog, dna)
Flipkart, Focus, and Free Advice – Shipping Charges Also Waived!What is one to make of a statement like this - “India is not mobile-first, but mobile-only country[1]”? Especially so if it is from the co-founder of the largest ecommerce company in India, and it turns out the company does not even have an app for the Apple iPad?

I have written at length on the distractions that seem to have been plaguing Flipkart and why it cannot afford to drop its guard in this fiercely contested space[2] - especially in light of all the noise surrounding its mobile ambitions. Somewhat paradoxically, this post is about offering advice to Flipkart that calls for some diversification!

As a logical next step, I wanted to take a look at Flipkart’s mobile apps – both on the iOS and Android platforms – to see how well they were executing on their very bold ambitions. As an aside, I also wanted to see if these (and competitive) mobile apps were leveraging all the computing power now available on tap inside these tiny devices. After all, apart from the recent – and amazing – advances Google has made in its voice-based search capabilities[3], there was this stunning demo from Hound[4] that gave a glimpse into the huge advances that voice-recognition, search, and machine-learning technologies have made in the last decade.
#MustRead - what next, e-tailing ? @flipkart Flipkart, Focus and Free Advice http://t.co/zincR6LS9D via @dna @AbhinavAgarwal
— Harini Calamur (@calamur) June 28, 2015
Flipkart, Focus and Free Advice http://t.co/UNF20JCqer by @AbhinavAgarwal
— dna (@dna) June 27, 2015The results were, to put it mildly, massively disappointing – which I will describe in some detail.
It should be clear that Amazon and Flipkart and SnapDeal are going to be at each other’s throats in the Indian online retail market. This is one battle from which neither player can walk away. Amazon has lost the China market to Alibaba (“In the first quarter of 2014, Alibaba's e-tailing site had a 48.4 per cent market share against Amazon China's less than 3 per cent.”[5] If that was not enough, Alibaba and Foxconn are in talks with SnapDeal for a rumoured $500 million investment![6]).

Amazon cannot afford to now lose the India market to a local upstart. Flipkart, on the other hand, has even less choice. It plays only in the Indian market. It cannot walk away either; there is no other market for it to walk towards. Its valuations – expected to rise to $15 billion after its next round of funding[7] make it way too costly for it to be acquired – at least profitably so for those funders who have put in hundreds of millions of dollars at these later and higher valuations. Amazon and Flipkart have deep pockets; Flipkart can afford to bleed hundreds of millions of dollars a year even as it grows, while Amazon has conditioned Wall Street to grant it the currency of ultra-high valuations even as it operates on razor-thin margins. It is unlikely that either will be able to deliver a knockout punch to the other anytime soon. This is a fifteen-round slugfest that will be decided by who can keep soaking in the blows and keep standing at the end of the fifteenth round; while they fight, the customer continues to win. Amazon has more diversity in its portfolio of business divisions than does Flipkart – ecommerce, cloud computing services, streaming audio and video, MRO and industrial supplies, smartphones, tablets, and more. While these divisions may at times face off against each other in expectedly healthy and sometimes unhealthy rivalry, they still form a formidable front against the competition. To quote these immortal lines from the Mahabharata, “we may be five against a hundred, but against a common enemy we are a hundred and five.”

So what does Flipkart do? Three things, to begin with.

First, it needs to get serious about software.
When you have a web site that offers millions of products from tens of thousands of resellers to millions of customers that reside in tens of thousands of cities and towns and villages, you need to make sure that your customers are seeing the products that are of most relevance to them, and which they are most likely to buy. If that problems looks like a nail to you, specifically a large-scale optimization problem with a huge number of decision variables, then large-scale computing and regression modelling are the hammer. You need to be applying this hammer to the almost infinite number of nails in front of you, all day and all night long. This is what enables you to present an ever-relevant basket of products to your customers, which keeps them engaged when on your site, and which hopefully makes them buy more often than not. Flipkart needs to take a close, long, hard look at its search capabilities – about which I will talk later in this post – and its suggestions engine, because both are very subpar at this point. If it’s any consolation, while Amazon is certainly better in the search department, its capabilities in this area are nothing great either, yet. Where Amazon scores over its competitors – every single one of them - is its huge and ever-growing corpus of customer reviews. Flipkart probably recognizes the important of this corpus of customer reviews, but has run into rough weather over the expected problem of fake reviews[8].

For inspiration on where the trifecta of search, machine learning, and e-commerce could venture – with Big Data in tow - one can turn to the story of how the popular American TV game show “Jeopardy” became the battleground for IBM researchers to build upon their experience with Deep Blue (the computer that had beaten world chess champion Gary Kasparov in 1997[9]) and to build a computer that would defeat the reigning champion of Jeopardy. That happened in February 2011, after four years of work led by IBM researcher David Ferucci and “about twenty researchers”[10].
This required advances in machine learning and other esoteric concepts like LAT (Lexical Answer Type), IDF (Inverse Document Frequency), temporal and even geospatial reasoning.[11] A new suite of software and platforms, built on a concept called genetic programming (“a technique inspired by biological evolution”) has started to make its way into mainstream commercial applications.  The algorithm here “begins by randomly combining various mathematical building blocks into equations and then testing to see how well the equations fit the data. Equations that fail the test are discarded, while those that show promise are retained and recombined in new ways so that the system ultimately converges on an accurate mathematical model.”[12] What this essentially means is going beyond keyword search-based correlations and moving to more semantic-oriented searches that combine machine learning with natural language processing. This in turn requires serious software brains (smart programmers using and refining the right algorithms and models) and muscle (massive learning and training sets in the hundreds of gigabytes running on clusters of tens of thousands of nodes).
If Flipkart is serious about the mobile ad business (about which I have expressed my reservations), even then it needs to get to the holy grail of deep-learning in ad-tech – “Inferring Without Interfering” the customer’s intent.”[13] In any event, this policy will only stand Flipkart in good stead. If they are already doing so, then good, but the proof is not in the pudding as much as in the eating of the pudding.

A critical differentiator in the coming times is not, I repeat, not, going to be driven by slick UIs or gimmicks on mobile apps like “shake to see offers”, but by offering truly intelligent and immersive experience that are feasible even today. Advances in machine learning, and capabilities such as voice, video, location, and more, when used in tandem will power the next set of innovations. Rather than stick to the tried and tested and old way of making users search using simple keywords and correlations and prior history, e-tailers need to make the shopping experience more intelligent.

Appendix 2 and 3 outline possible use-cases. It should be clear that both Flipkart and Amazon have a long, long way to go before realizing anything close to the vision outlined, but without such advances, competitors like Google will find the wedge they need to prise open this market for themselves.

Second, Flipkart (or even Amazon for that matter, or SnapDeal, or whichever competitor you happen to care about, though in this case the admonition is more targeted at Flipkart in light of its mobile-only pronouncements) needs to get serious about the mobile platform.

Browse to either Flipkart or Myntra’s websites from a browser on an iPad and you are asked to use their app instead. Would you believe if I told you Flipkart does not have an iPad app (as of 15th June 2015)? No? Go check for yourself – I did! Ditto for Myntra (the online fashion retailer Flipkart acquired in 2014)! See Appendix 1 for what I found when I downloaded their apps on my iPad tablet. This would be comedically farcical if serious money weren’t riding on such decisions.

Third, Flipkart needs to get into the cloud business.

Yes, I am serious.

Let’s look at the competition – Amazon. It is the 800 pound gorilla in the cloud computing industry, where its offering goes by the umbrella of AWS (Amazon Web Services) and offers almost everything you could think of under the cloud – platform, infrastructure, software, database, email, storage, even machine learning, and much more. How gorilla-ish? “AWS offers five times the utilized compute capacity of the other 14 cloud providers in the Gartner Magic Quadrant. Combined.[14]” Since 2005, Amazon has spent “roughly $12 billion” on its infrastructure[15]. It competes with the likes of Microsoft and Google in this space. Yet, Amazon’s cloud revenues are estimated to be “30 times bigger than Microsoft’s.[16]

And yet I argue that Flipkart should get into the cloud business. As I wrote last year[17], Flipkart had to invest substantially (per my estimates, more than one hundred crore rupees, or somewhere in the vicinity of $15-$20 million dollars – which is not chump change) to build its capacity to stand up to the traffic it expected for its “Big Billion Day”. This is in addition to the regular additions it must be making to its computing infrastructure. All this is not surprising, given that the retail business is prone to lumpiness in traffic – a disproportionate amount of traffic is concentrated around sale events, or holidays.

For example, while Amazon reportedly had ten million Prime subscribers in March 2013, it reported that over 10 million “tried Prime for the first time” over the holidays in 2014 (traditionally the period between Thanksgiving and Christmas).[18] To prevent web sites from keeling over under the crush of holiday traffic, companies invest substantially, in advance, to make sure the web site keeps chugging along. The flip side is that for those periods when traffic is more average and a fraction of peak traffic, all those thousands of computers, the hundreds of gigabytes of memory, terabytes of disk space, and gobs of network bandwidth capacity are lying idle – depreciating away, obsolescing away.

Amazon realized this a decade ago and started building a rental model around its excess capacity – this was the genesis behind Amazon Web Services. There is no reason for Flipkart to not do the same. What works for Amazon has worked quite well for Flipkart[19]. If it spins off its entire e-commerce infrastructure into a separate entity, it can palm much off the capital costs of its computing infrastructure to the cloud computing subsidiary, substantially improving its balance sheet in the process. You could argue this is nothing but an accounting gimmick, and I am not going to argue with that aspect of the decision - there would be undeniable and real benefits to this decision, and it’s childish to expect a business to be run on utopian principles. As things stand, the state government of Telangana is already assiduously wooing Amazon to invest in an AWS centre in the state[20]. Once operating on Indian soil, Amazon will be able to meet legal requirements that require certain categories of data to remain with the national borders.

Any industry so heavily influenced and shaped by technology as the e-commerce industry would do well to listen to the winds of change. If unheard and unheeded, these winds of change turn into gale storms of disruption that blow away incumbents faster than you can imagine. “Mobile-only” is a useful-enough mantra, but translating that into an “app-only” sermon hints at myopic thinking – a troubling sign for sure. It turns out that Google “secretly” acquired a company that specializes in “streaming native mobile apps”. Is this a shape of the things to come? How will this transform the world of mobile apps, or even the mobile landscape in general? Time will tell, but “lock-in” may well be a wise strategy for your customers, but a terrible one to apply to yourself.[21].

Appendix 1 - App-solutely Serious about Apps?Fire up your favourite mobile browser on an Apple iPad and browse to Myntra’s website (that would be www.myntra.com). You are greeted with a message to vamoose to their mobile app, because after all, Myntra is all about mobility – social mobility in fashion, and mobile devices when speaking more literally.
Figure 1 - Myntra web site on tablet browser

Incredulity hits you in the face when you realize that (on the Apple App Store) the Myntra app is “optimized for iPhone 5, iPhone 6 and iPhone 6 Plus”, but not the iPad. Yes, you read that right – the web site that tells you have to use its mobile app and mobile app only on an iPad does not have an app optimized for the iPad.
Figure 2 - Myntra app details on the Apple App Store

I am, however, somewhat of a cynical person. I tried searching for the keyword “myntra” on the Apple App Store. The only filter applied was to look for “iPad Only” apps. Here are the beatific search results. Indian gave the world the concept of zero, and the search results page gave one practical application of that elegant mathematical concept.
Figure 3 - Search results for "iPad Only" apps on the Apple AppStore for "myntra"

So where was that Myntra app hiding? I changed the filter to “iPhone Only”, and true-enough, there was that Myntra app.
Figure 4 - Myntra app on the Apple App Store

In case you are wondering how that was even possible, know that most apps created for the iPhone (or iPod Touch) can run on an iPad without any modifications – all that is required for you to keep this in mind when compiling the app. Apple calls this a “Universal app”[22].

Now that can’t be so bad, right? After all, the app is available on the iPhone and the iPad, so where and what is the grouse? I will come to that in just a bit, but take a look at what the Myntra app looks like when run on the iPad.
Figure 5 - Myntra app running on an iPad
This is how the app runs inside an iPad. You have the option of tapping the “2x” button, after which the app uses the full screen, but by scaling everything to twice its size. There is no other intelligence here being applied – like changing the icons, or the text, or adding more features. This is iOS doing what little work you see.
Why this arouses incredulity is due to the stunning dissonance one experiences – between the statements of the Myntra (and Flipkart) executives going to town about a “mobile-only” world[23] on the one hand and the reality of a missing-in-action iPad-optimized app on the other. Yes, one could make the argument that Apple commanded a stunningly low single-digit share of 7% of the tablet market in India[24], but to make this argument is to negate your very philosophy of a “mobile-only” world. Mobile includes smartphones, tablets, phablets, wearables (for which Flipkart does have an app![25]), smart-TVs, and even embedded devices.
Flipkart’s mobile web site works - at least for now - on the iPad (though it does not on a smartphone – you have no option but to use their app), but the story is not much different there. No iPad-optimized app, but a smartphone app that does duty on the iPad by virtue of it being a “Universal” app.
Figure 6 - Flipkart shopping app in the Apple App Store

 
Figure 7 - Flipkart shopping app on the Apple iPad
 It’s not as if Amazon’s iPad app is much better. Yes, they do have an iPad app, but it looks more like a hybrid app – a native shell with an embedded browser snuck in, and very little by way of any tablet optimizations.
Figure 8 - Amazon app for the iPad

Appendix 2 – Natural Speech SearchesMobile shopping apps like Flipkart and Amazon provide you the option of inputting your search query via voice (more because of the support the underlying mobile OS provides), but that forces you say out aloud what you have typed – keywords, and nothing more.
Unlike the stunning Hound demo or the capabilities of Google Now[26], e-tailers have yet to leave the stone age in search capabilities. While Hound can understand and answer (correctly) queries like “Show me hotels in Seattle for Friday, staying one night” and then support refinements to the query like “Show only the ones costing less than $300” or “Show only the ones that have three or four or five stars that are pet friendly, that have a gym and a pool, within 4.5 miles of the Space Needle”[27], and Google Now can understand foreign accents (like my Indian English accent) and parse phrases like “ghat”, “jyotirling” and more, a relatively simple phrase like - “What are the best sellers in fiction” – leads to disappointment on both Amazon and Flipkart’s mobile apps.
Figure 9 - Search results in the Amazon app
And to be clear, what was presented was not the bestsellers list, because the bestseller list looked like this:
Figure 10 - Non-fiction bestsellers in books as shown on the Amazon app

I tried another search – “Suggest books for children”. I don’t know what to call the search results, but one with “*divorce* your Child” as the first result is surreal.
Figure 11 - Search results on Amazon app

To complete my brief experiment on Amazon, I tried “Show me best sellers in electronics”. That also did not yield any relevant results.
Figure 12 - Search results in the Amazon app
Flipkart is not much better, and at this point we are really looking at rock-bottom as the baseline. Even a marginal improvement would be welcome here. Sadly, not the case. Though, Flipkart does separate each word out and allow you to delete any one word to refine your search. Given the abysmal quality of search results, it is somewhat of a zero-divide-by-zero case, resulting in only infinite misery trying to find the right combination of keywords that will yield the desired results.
Figure 13 - Search results on the Flipkart app

Does the Myntra app fare any better? Predictably, it doesn’t. If semantic search in the e-commerce space was a problem that had been cracked by either Flipkart or Myntra, it would have been shared across both platforms by now.
Figure 14 - Search results in the Myntra app

Even Google, with its oft-stated e-commerce ambitions[28],[29] , and the eye-popping advances that it has made with its voice-based search (Siri from Apple and lately Cortana from Microsoft also deserve to be included, but neither company seems to be quite interested in e-commerce at the scale of Amazon, yet) left me disappointed with a simple search like – “what are the fiction best sellers in India”.
Figure 15 - Search results in the Google app

Appendix 3What do I have in mind with respect to the kinds of queries that Flipkart (or Amazon) should be trying to enable? Without any further context, I present the following examples:
One:
(this is a comparatively simpler form of the semantic search capabilities I propose)
Me: show me the best sellers in non-fiction
App: [displays a list of book best sellers in non-fiction] [Optionally, excludes or places the ones I have bought at the bottom of the list; or marks them differently and provides me with an option of reading them online – assuming I had purchased an e-book version]
Me: show me only those books that have been published in the last three months;
App: [filters the previous set of search results to show only those non-fiction best sellers that have been published in the last three months]
Me: also include books that were on the bestseller list this year
App: [adds books that were in the top 10/20 bestsellers list in 2015 but have now dropped out of the rankings]
Me: cancel the last search, and show me those books that are also available as e-books, and then sort them by price
App: [displays a list of book best sellers in non-fiction, filtered by those available on the Kindle, and sorts by price, ascending]
Me: send me free e-book samples of the first five books from this list and remind me in one week whether I want to purchase them.
App: [downloads free samples of the first three books to my e-book app] [creates a reminder to remind me in one week]


Two:
(this is a more social and more nuanced form of the semantic search outlined above)
Me: show me a list of LED TVs
App: [displays a list of the bestselling LED TVs]
Me: show me LED TVs that are HD, 40 inches or larger, cost no more than Rs 60,000, and can be delivered in the next three days.
App: [displays a list of TVs matching the criteria, and adds – “there are only three TVs that match your search criteria, so I have changed the price to Rs 70,000, which has resulted in five more search results. Say “cancel” to undo.”]
Me: Which among these would be most relevant to me?
App: [displays the list sorted based on popularity in my postal code] [offers to show the list sorted on TVs sold in the last three months to the housing community I live in – or the company I work at – or based on people with my profile of educational qualifications or marital/family status – based on privacy settings of course]
Me: summarize the most useful reviews for the first TVs, and keep each under two minutes.
App: [summarizes the most useful reviews and then reads out a software-generated summary, in less than two minutes. Also sends a text summary to my WhatsApp or email]
Far-distant utopia? Naah, I don’t think so. This is within the realm of the possible, and I expect to see this become reality in the next two years. Today, however, we are some ways off from the innovations where online shopping will become a truly immersive, interactive experience akin to having a natural conversation with an incredibly knowledgeable yet infinitely patient salesperson.

Three:
(ratcheting things up one more notch)
Me: (standing amidst the ruins of Hampi) Suggest some good books about this place.
App: [suggests bestsellers or highest-rated books on three categories: coffee-table books on Hampi; history of Hampi and the Vijayanagar Empire; historical fiction books set in the fifteenth/sixteenth century Vijaynagara Empire]
Me: Also suggest something on the significance of this chariot temple
App: …

Four:
App: [reminds me that I have a party at my house this weekend where four families are coming over]
Me: I need some snacks and also suggest some recent action movies to rent
App: [suggests food-items to order and shows a list of the five top grossing movies of the year in the “Action” genre and shows options: buy, rent (really?), stream]
Me: place the first, third, and fifth items in the shopping cart, record this and deliver to my wife. Then rent to stream the third movie in HD format on Saturday evening.
App: [places these items in the shopping cart, records a 15 second video and pings the spouse via a notification/alert to view the video. It also places an order for the selected movie]

Disclaimer: views expressed are personal.

 References:
[1] "India is not mobile-first, but mobile-only country: Sachin Bansal, Flipkart's founder and Mukesh Bansal, Myntra's CEO - timesofindia-economictimes", http://articles.economictimes.indiatimes.com/2015-05-13/news/62124447_1_myntra-sachin-bansal-ceo-mukesh-bansal
[2] See http://www.dnaindia.com/analysis/standpoint-flipkart-vs-amazon-beware-the-whispering-death-2079185 and http://www.dnaindia.com/analysis/standpoint-why-flipkart-seems-to-be-losing-focus-2076806
[3] "Google Launches Custom Voice Actions For Third Party Apps", http://searchengineland.com/google-launches-custom-voice-actions-for-third-party-apps-220148
[4] "After Nine Years of Secret Development, Hound Voice Search App Has a Dazzling Demo | Re/code", http://recode.net/2015/06/02/after-nine-years-of-secret-development-hound-voice-search-app-has-a-dazzling-demo/
[5] "A missed opportunity in China has Amazon founder Jeff Bezos backing his India venture", http://indiatoday.intoday.in/story/amazon-jeff-bezos-india-business-venture-flipkart-snapdeal/1/393933.html
[6] "Alibaba, Foxconn in Talks to Invest $500 Million in India’s Snapdeal - India Real Time - WSJ", http://blogs.wsj.com/indiarealtime/2015/06/16/alibaba-foxconn-in-talks-to-invest-500-million-in-indias-snapdeal/
[7] "Flipkart set to raise up to $800 million - Livemint", http://www.livemint.com/Companies/j2B9ax1SNS4JrDdJAU19sO/Flipkart-set-to-raise-up-to-800-mn.html
[8] See "How e-retailers such as Flipkart, Amazon are keeping fake products at bay - timesofindia-economictimes", http://articles.economictimes.indiatimes.com/2015-01-08/news/57791521_1_amazon-india-sellers-mystery-shoppers, "Who Reviews the Reviewers? How India's Online Businesses Are Fighting Fake Reviews | NDTV Gadgets", http://gadgets.ndtv.com/internet/features/who-reviews-the-reviewers-how-indias-online-business-are-fighting-fake-reviews-697112, and "How genuine are product reviews on FlipKart? - Quora", http://www.quora.com/How-genuine-are-product-reviews-on-FlipKart
[9] "Deep Blue (chess computer) - Wikipedia, the free encyclopedia", https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)
[10] "Rise of the Robots: Technology and the Threat of a Jobless Future”, by Martin Ford, Jeff Cummings, ISBN 9781480574779, http://www.amazon.com/Rise-Robots-Technology-Threat-Jobless/dp/1480574775
[11] Ibid and "The AI Behind Watson — The Technical Article", http://www.aaai.org/Magazine/Watson/watson.php
[12] "Rise of the Robots: Technology and the Threat of a Jobless Future”
[13] Gregory Piatetsky on Twitter: "The #DeepLearning future of ads: "Inferring Without Interfering with what moves the Customer most" - J. Kobelius http://t.co/zh96DC6DDG" https://twitter.com/kdnuggets/status/610848069672927232
[14] "Gartner: AWS Now Five Times The Size Of Other Cloud Vendors Combined - ReadWrite", http://readwrite.com/2013/08/21/gartner-aws-now-5-times-the-size-of-other-cloud-vendors-combined
[15] Ibid.
[16] "How much bigger is Amazon’s cloud vs. Microsoft and Google?", http://www.networkworld.com/article/2837910/public-cloud/how-much-bigger-is-amazon-s-cloud-vs-microsoft-and-google.html
[17] "A Billion Dollar Sale, And A Few Questions", http://www.dnaindia.com/analysis/standpoint-a-billion-dollar-sale-and-a-few-questions-2047853
[18] "Amazon added 10M new Prime subscribers over the holidays, could make up to $1B in annual revenue | VentureBeat | Business | by Harrison Weber", http://venturebeat.com/2014/12/26/amazon-made-nearly-1b-from-new-prime-subscriptions-over-the-holidays/
[19] See http://www.dnaindia.com/analysis/standpoint-flipkart-vs-amazon-beware-the-whispering-death-2079185
[20] "Amazon likely to bring Web Services to Telangana | Business Standard News", http://www.business-standard.com/article/companies/amazon-likely-to-bring-web-services-to-telangana-115061000659_1.html
[21] "Report: Last Year Google Secretly Acquired Agawi, A Specialist In Streaming Native Mobile Apps | TechCrunch", http://techcrunch.com/2015/06/18/report-last-year-google-secretly-acquired-agawi-a-specialist-in-streaming-native-mobile-apps/
[22] "Start Developing iOS Apps Today: Tutorial: Basics", https://developer.apple.com/library/ios/referencelibrary/GettingStarted/RoadMapiOS/FirstTutorial.html
[23] http://articles.economictimes.indiatimes.com/2015-05-13/news/62124447_1_myntra-sachin-bansal-ceo-mukesh-bansal
[24] "Samsung Tops Indian Tablet Market Share, Followed By Micromax, iBall", http://trak.in/tags/business/2014/11/28/indian-tablet-market-share-growth/
[25] "Flipkart launches an app for Android Wear sporting wearables - Tech2", http://tech.firstpost.com/news-analysis/flipkart-launches-an-app-for-android-wear-sporting-wearables-240070.html
[26] See this for a comparison between Hound and Google Now - "Here’s how Hound beta compares to Google Now (Video) | 9to5Google", http://9to5google.com/2015/06/05/hound-beta-vs-google-now-video/
[27] "After Nine Years of Secret Development, Hound Voice Search App Has a Dazzling Demo | Re/code", http://recode.net/2015/06/02/after-nine-years-of-secret-development-hound-voice-search-app-has-a-dazzling-demo/
[28] "Google Preps Shopping Site to Challenge Amazon - WSJ", http://www.wsj.com/articles/google-preps-shopping-site-to-challenge-amazon-1418673413
[29] "Google Finds Partners To Help It Compete With Amazon - Forbes", http://www.forbes.com/sites/benkepes/2015/04/13/google-finds-partners-to-help-it-compete-with-amazon/



© 2015, Abhinav Agarwal. All rights reserved.

Oracle Priority Support Infogram for 02-JUL-2015

Oracle Infogram - Thu, 2015-07-02 14:53

Oracle Support

Two good items from the My Oracle Support blog:




MySQL

MySQL Enterprise Monitor 3.0.22 has been released, from the MySQL Enterprise Tools Blog.

Big Data


WebLogic

Additional new material WebLogic Community, from WebLogic Partner Community EMEA.

Improve SSL Support for Your WebLogic Domains, from Proactive Support - Identity Management.

Fusion Middleware

Calling Fusion SOAP Services from Ruby, from Angelo Santagata's Blog.

JDBC


OBIEE

OBIEE SampleApp V506 is Available, from Business Analytics - Proactive Support.

Ops Center

Upgrading to 12.3, from the Ops Center blog.

Identity Management

Configuring OAM SSO for ATG BCC and Endeca XM, from Proactive Support - Identity Management.

And from the same blog: Monitoring OAM Environment

Health Sciences


Primavera

New Primavera P6 Release 15.1 Patch Set 2, from the Primavera Support Blog.

EBS

From the Oracle E-Business Suite Support blog:








Query existing HBase tables with SQL using Apache Phoenix

Kubilay Çilkara - Thu, 2015-07-02 14:25
Spending a bit more time with Apache Phoenix and reading again my previous post I realised that you can use it to query existing HBase tables. That is NOT tables created using Apache Phoenix, but HBase - the columnar NoSQL database in Hadoop.

I think this is cool as it gives you the ability to use SQL on an HBase table.

To test this, let's say you login to HBase and you create an HBase table like this:

> create 'table2', {NAME=>'cf1', VERSIONS => 5}

The table2 is a simple table in HBase with one column family cf1 and now let's put some data to this HBase table.

> put 'table2', 'row1', 'cf1:column1', 'Hello SQL!'

then maybe add another row

> put 'table2', 'row4', 'cf1:column1', 'London'

Now, in Phoenix all you will have to do is create a database View for this table and query it with SQL. The database View will be read-only.  How cool is that, you don't even need to physically create the table or move the data to Phoenix or convert it, a database view will be sufficient and via Phoenix you can query the HBase table with SQL.

In Phoenix you create the view for the table2 using the same name. As you can see below the DDL used to create the view is case sensitive and if you created your HBase table name in lower case you will have to put the name in between double quotes.

So login to Phoenix and create the "table2" view like this:

> create view "table2" ( pk VARCHAR PRIMARY KEY, "cf1"."column1" VARCHAR );

And here is how you then query it in Phoenix:


SQL Query on Phoenix
Tremendous potential here, imagine all those existing HBase tables which now you can query with SQL. More, you can point your Business Intelligence tools and Reporting Tools and other tools which work with SQL and query HBase as if it was another SQL database.

A solution worth investigating further? It definitely got me blogging in the evenings again.

To find out more about Apache Phoenix visit their project page https://phoenix.apache.org/



Categories: DBA Blogs

Streamline Oracle Development with Cloud Services

Angelo Santagata - Thu, 2015-07-02 05:24
Streamline Java Development with Cloud Services
On-Demand Webinar Replay

Learn to deliver java applications to market faster. Reduce hardware and software costs for new development and testing environments. Improve DevOps efficiency. Build, test and run enterprise-grade applications in the Cloud and on premise.

Listen to this webinar replay with development expert James Governor, co-founder of RedMonk, and Daniel Pahng, President and CEO of mFrontiers, LLC, an ISV with hands-on experience developing enterprise mobility and Internet of Things (IOT) solutions, as they present this webcast on developing applications in the cloud.
Listen today!
For more information:
July 2015                                                                                          Oracle Corporation - All rights reserved

Calling Fusion SOAP Services from Ruby

Angelo Santagata - Wed, 2015-07-01 08:55
Just completed some integration work with a partner of ours using the Ruby language. Given that a lot of startups like Ruby I thought it would be useful to cut-n-paste the sample code here.
This example creates a simple (minimal) opportunity using the SOAP API in Sales Cloud. That said the code would be almost identical if you were querying HCM Data,
The approach we took here was to prototype the SOAP call using SOAPUI and then cut-n-paste the SOAP payload into the data variable. In a real industrialized solution I'd create the payloads in template form.


  def create_opportunity

  # Change yourhostname.com to your Fusion SOAP Endpoint Hostname

    http = Net::HTTP.new(uri.host, uri.port)
    http.use_ssl = true
    path = uri.request_uri
    http.read_timeout = 5
    http.open_timeout = 5


  # Change authorization header to contain Base64encoded string of username/password
    headers = {
    'Content-Type' => 'text/xml',
    'authorization' => 'Basic bBase64EncodedCredentialHere='
    }

   
   # Data Contains the payload
    <soapenv:Header/>
    <soapenv:Body>
    <typ:createOpportunity>
    <typ:opportunity>
    <opp:Name>Joel Test New1</opp:Name>
    </typ:opportunity>
    </typ:createOpportunity>
    </soapenv:Body>
    </soapenv:Envelope>'

    resp, data = http.post(path, data, headers)
  end

!TIP!
A quick test in a SOAP testing tool like JDevelopers Http Analyzer or SOAPUI is a MUST before executing this!

Apache Phoenix, SQL is getting closer to Big Data

Kubilay Çilkara - Tue, 2015-06-30 16:50


Here is a post about another project in the Big Data world, like Apache Hive from my previous post, enables you to do SQL on Big Data. It is called Apache Phoenix.

Phoenix is a bit different, a bit closer to my heart too, as I read the documentation on Apache Phoenix, the word 'algebra' and 'relational algebra' came across few times, and that mean only one thing, SQL! The use of the word algebra in the docs did give me a lot of confidence. SQL has closure, is based on a database systems model which has it's roots in logic and maths and especially a subset of algebra, The Set Theory.

Apache Phoenix is developed in Salesforce and is now one of the popular projects in Apache. Apache Phoenix is a SQL skin on top of HBase, the columnar (NoSQL) database of the Hadoop ecosystem, capable of storing very large tables and data and query them via 'scans'. HBase is part of the Hadoop ecosystem and the file system it uses is usually HDFS. Apache Phoenix is using JDBC on the client as a driver.

In the race to bring the easiest to use tools for Big Data, I think Apache Phoenix is very close. It is the SQL we know used since the 1970s. The Apache Phoenix team seems to be committed and willing to introduce all of the missing parts of SQL, including transaction processing with different isolation levels.  Making Phoenix a fully operational Relational Database layer on HBase. Have a look in their roadmap. The amount of current and suggested future SQL compatibility is remarkable, and this makes me take them really seriously.
  • Transactions
  • Cost-based Query Optimization! (Wow)
  • Joins
  • OLAP
  • Subqueries
  • Striving for full SQL-92 compliance
In addition to all this, it is also possible to turn an existing HBase table to an Apache Phoenix table using CREATE TABLE or even CREATE VIEW, the DDL statements that we know. How handy is that? Suddenly you can SQL enable your existing HBase database!

How to install and use Phoenix

The SQL skin can be installed to an existing Hadoop HBase installation very quickly. All you need to do is to download and extract the tarball. You can setup a standalone Hadoop environment, look at my previous blog post for that, and then install HBase and install Apache Phoenix

Once the Apache  Phoenix software is installed, then you can start it and query it with SQL like this.

From within the bin/ directory of Phoenix install directory run

$ ./sqlline.py  localhost

That will bring you to the phoenix prompt


0: jdbc:phoenix:localhost> select * from mytable;


Categories: DBA Blogs

New Tools releases , now with Java

Kris Rice - Tue, 2015-06-30 11:28
What's New   For the 90%+ of people using sqldev/modeler on windows, the JDK bundled versions are back.  So no more debating what to install or worrying about conflicting Java versions.   Lots of bug fixes.    My favorite bug is now fixed so you can use emojis in your sql> prompt. RESTful CSV Loading   We wrapped he same CSV import code in SQL Developer into the REST Auto-Enablement

Interaction Hub Image Now Available on the PeopleSoft Update Manager Home Page

PeopleSoft Technology Blog - Tue, 2015-06-30 11:23

As noted in a recent post, the PeopleSoft Interaction Hub is now part of the Selective Adoption Process.  You can get the first image now on the PUM home page.  (At the PUM home page, choose the PeopleSoft Update Image Home Pages tab, then select the Interaction Hub Update Image page from the drop down.)  This means customers can use the PeopleSoft Update Manager and our other life cycle tools to manage their upgrade and maintenance process for the Hub.  There is also a white paper posted there that describes the baseline customers must reach to start taking these images. 

Note that this will be the only way for customers to take maintenance and updates going forward, so we encourage everyone to move to the Selective Adoption process as soon as is feasible for your organization. This move brings the Interaction Hub in line with all other PeopleSoft applications, which use the Selective Adoption process.  This process also offers customers additional value and control, and enables you to benefit from the value of the latest features with a greatly streamlined life cycle process.  

For customers that are eager to learn more, there are many resources on Selective Adoption and PUM on the PUM home page as well as on our YouTube channel.

This first image of the Interaction Hub is functionally equivalent to the current release (9.1/Revision 3), but taking it gets you on the Selective Adoption process.  Some great enhancements are coming the the next image.


Make even more of UKOUG Tech15: APEX 5.0 UI Training - Dec 10th in Birmingham

Dimitri Gielis - Tue, 2015-06-30 01:48

APEX 5.0 has been released this spring. People who have already spent some time on this new version know this version is packed with new features aimed to make APEX developers even more productive, like the Page Designer.
Another striking new subset of features is aimed at creating better looking user interfaces for your APEX applications in an easy and maintainable way. 
The definition of user interface components in APEX 5.0 is very different to what we're used to. For example there is a new Universal Theme with Template Options and a Theme Roller. To get you up and running with this new toolset as quickly as possible, Dimitri Gielis of APEX R&D and Roel Hartman of APEX Consulting have joined forces and set up a one day course fully aimed at APEX 5.0 UI. So if you want to know not only how to use the new Theme, but also how to modify it to fit your needs, this is the event you should attend!

The training will be at the Jury’s Inn in Birmingham (UK) on Thursday Dec 10 - so conveniently immediately after the UKOUG Tech15 conference.
More information and registration see www.apextraining.eu

If you are from another country and think this training should be available in your country as well, please contact us - then we'll see what we can do!
Categories: Development

ReConnect 2015

Jim Marion - Mon, 2015-06-29 18:43

It is just a little less than a month until the PeopleSoft ReConnect conference in Rosemont, Illinois. I will be presenting PeopleTools Developer: Tips and Techniques on Thursday from 11:30 AM to 12:20 PM in Grand Ballroom H.

Video Tutorial: XPLAN_ASH Active Session History - Part 6

Randolf Geist - Sun, 2015-06-28 16:30
The next part of the video tutorial explaining the XPLAN_ASH Active Session History functionality continuing the actual walk-through of the script output.

More parts to follow.

Apple Watch Impressions with Jeremy Ashley: Time for the Best User Experience in the Enterprise Cloud

Usable Apps - Sat, 2015-06-27 04:40

In part two of a three-part series, Ultan O'Broin (@usableapps) talks with Jeremy Ashley (@jrwashley) about his impressions of the Apple Watch and other insights during a day in the life of a Group Vice President in Oracle. Read part one.

"Perhaps it's an English thing,” says Oracle Applications User Experience Group Vice President, Jeremy Ashley, "but just being able to keep eye contact with someone when we're talking means I can pay closer attention to people."

 Inspiring Cloud UX in Oracle

Jeremy Ashley: Inspiring user experience leadership of strategy, science, and storytelling.

"A glance at my Apple Watch and I know immediately if something is important. I can decide if I need to respond or it can wait. I don't have to pull out my smartphone for that."

This story of combining the personal convenience of wearable technology with empathy for people is typical of the man who sets the vision for the Oracle Applications Cloud user experience (UX).

It’s just one of Jeremy's impressions of the iWatch, as it's known. Now that he's used the Apple Watch for a while since we first chatted, I wanted to find out about his experience and what it all means for enterprise UX.

iWatch iMpressions

"I just love the sheer build quality of the watch; so utterly Apple," Jeremy begins. His industrial design background surfaces, bringing together traditions of functionality, classic craftsmanship, and exuberance for innovation: "Sweet. I can even use it to tell the time!"

A bloke with an eye for pixel-level detail, Jeremy has explored how to get the best from the Apple Watch, right down to the exact precision needed for the force touch action on the built-in Maps app. He's crafted a mix of apps and favorite glances to suit his world, such as for battery life, his calendar, and stocks. He admires the simplicity and visualizations of the built-in Activity app too, swiping the watch face to see his latest progress as we talk in his office full of what's hot in technology and a selection of clocks and traditional woodworking tools.

Microtransactions at a glance from the wrist

Microtransactions at a glance from the wrist delight the wearer and make life—and work—more convenient.

"The watch really shows how the idea of context automates the routine and looks after the little things that make life easier and delight you in simple ways, such as not having to swipe a credit card to pay for coffee."

In the enterprise world, these kinds of little experiences, or "microtransactions" as Jeremy calls them, translate to wearer convenience when working. For example:

  • Automatically recording the time spent and location of a field service job
  • Accepting terms and conditions when attending a confidential demo meeting as you check in at reception
  • Adding data to the cloud, such as updating a win probability as you walk away from a sales engagement
  • Seeing at a glance that a supply chain fulfillment is complete

Oracle Glance and the Enterprise

Oracle's concept of glance is device agnostic and reflects a key UX strategy—mobility—or how we work flexibly today: across devices, pivoting around secure enterprise data in the Oracle Cloud.

"Smartwatches are like mobile dialog boxes," Jeremy explains. "They start that user conversation with the cloud in simple, 'in-the-moment,' deeply contextual ways. Glance and the cloud together automatically detect and deliver the who, what, and where of microtransactions, yet because it's all on a watch, the experience remains personal and familiar. That really resonates with wearers."

The smartwatch is a personal and familiar paradigm in the enterprise too

Jeremy Ashley: The smartwatch is a personal and familiar paradigm that also resonates in the enterprise.

Jeremy shared some thoughts on where such innovation is heading:

"The Apple Watch won't replace the smartphone, for now. We still need that identifier device—a kind of personal beacon or chip, if you like—that lets us make an elegant 'handoff' from a glance on our wrist to a scan for denser levels of information or to a commit to doing less frequent tasks on other devices. The watch just isn't designed for all that."

Activity app Stand goal glances on Apple Watch

Apple Watch Activity glances for stand goal progress

A perfect example of Oracle Cloud UX strategy and design philosophy together. Jeremy glances back at his Activity app and sees his new stand goal progress. That standing desk is paying off!

But, innovating user experience in Oracle is an activity that definitely does not stand still. We'll explore how such innovation and design progress pays off for enterprise users in a future blog post.

Got Time Now?

Discover more:  

New PeopleTools Mobile Book

Jim Marion - Fri, 2015-06-26 19:44

My wife and I have been writing another book. We are reviewing proofs now, which means we are getting close to publication. I did a quick search on Amazon and see that Amazon is taking pre-orders: PeopleSoft PeopleTools: Mobile Applications Development. Publication date is currently set for October 16, 2015, which means it will publish just before OpenWorld. Fluid and MAP have been out for about a year. If you guessed that a new PeopleTools mobile book would cover these mobile technologies, you guessed correctly. But I saw no reason to stop there. After describing how to use Fluid and MAP, the book moves on to building responsive mobile applications using standard HTML5 development tools and libraries including jQuery mobile and AngularJS. Just today I spoke with a customer still using PeopleTools 8.50. What are the odds that customer will be using PeopleTools 8.54 in the next year? The second section of this book, using HTML5 is perfect for a customer in this situation because it describes how to connect a modern single page application to a PeopleSoft back end using iScripts and REST services (one chapter for each back end solution). The book finishes with examples of building native and hybrid applications for PeopleSoft using the Android SDK, Apache Cordova (my personal favorite), and Oracle's Mobile Application Framework. Here is a rough outline:

Chapter 1 shows you how to prepare your workstation for mobile development. This includes configuring HTML5 browsers, developer tools, and emulators.

Chapter 2 digs into Fluid, showing two examples of creating Fluid pages. The first is a basic page whereas the second is a two-column responsive page. This chapter covers search pages; toolbar actions; and fluid field, page, and component features. The point of this chapter is to help the reader feel comfortable with Fluid. Fluid includes a lot of new features based on HTML5, CSS3, and JavaScript. I really want customers to understand, however, that they can build Fluid pages using core PeopleTools without any knowledge of the modern web concepts. Of course, you can build some really amazing solutions if you know HTML5, CSS3, and JavaScript.

Chapter 3 explains the new Mobile Application Platform (MAP): what it is, when to use it, and how to use it. A chapter wouldn't be complete without examples, so there are plenty of examples to help you start your next MAP project.

Chapter 4 segues into modern mobile development. The rest of the book takes the user interface outside of PeopleTools. Before moving away from Application Designer, however, we need a data model and a scenario. This chapter presents the scenario and lays the foundation for the rest of the chapters. In this chapter you will work with SQL and the Documents module.

Chapter 5 shows us how to create our first HTML5 front end. I wanted to make this chapter as simple as possible so I used jQuery Mobile. In this chapter the reader will write very basic HTML and have the opportunity to see how jQuery Mobile progressively enhances simple markup to create impressive mobile solutions.

Chapter 6 is the exact opposite of chapter 5. Here I wanted to demonstrate flexibility and performance. This chapter is intentionally designed to provide a challenge for developers. Readers tell me it is a good chapter, perhaps a little intimidating, but very worthwhile. In this chapter you will work with AngularJS, Topcoat, and FontAwesome.

Chapter 7 shows the reader how to build back-end services for Chapters 5 and 6 using iScripts.

Chapter 8 is the same as Chapter 7 but uses REST services instead of iScripts. If you are new to PeopleSoft REST services and want to learn how to configure REST services as well as how to work with Documents to serve JSON from Integration Broker, then you will find this chapter very valuable.

Chapter 9 shifts from HTML5 to native. In this chapter the reader will learn how to use the Android SDK to consume the services built in chapter 8. The point of this chapter is not to teach Android development but rather how to consume PeopleSoft services from Android.

Chapter 10 turns to a native application category described as Hybrid applications. In this Chapter the reader will learn how to convert the Chapter 6 prototype into an on-device application that has access to device specific features such as the camera. In fact, the example shows how to use the Cordova API to take a selfie.

Chapter 11 brings us back to Oracle-specific technology by showing how to build a hybrid application using Oracle's Mobile Application Framework (MAF). I chose to spend a little more time in this chapter to teach some of the specifics of MAF. For example, I wasn't very excited about the default appearance of buttons on Android so I included steps showing how to extend the MAF skin.

Publication is still a few months away, but we are getting close. I'm really hoping to be able to give away copies during my OpenWorld session this year.

SQL*Plus tips #7: How to find the current script directory

XTended Oracle SQL - Fri, 2015-06-26 14:06

You know that if we want to execute another script from the current script directory, we can call it through @@, but sometimes we want to know the current path exactly, for example if we want to spool something into the file in the same directory.
Unfortunately we cannot use “spool @spoolfile”, but it is easy to find this path, because we know that SQL*Plus shows this path in the error when it can’t to find @@filename.

So we can simply get this path from the error text:

rem Simple example how to get path (@@) of the current script.
rem This script will set "cur_path" variable, so we can use &cur_path later.
 
set termout off
spool _cur_path.remove
@@notfound
spool off;
 
var cur_path varchar2(100);
declare 
  v varchar2(100);
  m varchar2(100):='SP2-0310: unable to open file "';
begin v :=rtrim(ltrim( 
                        q'[
                            @_cur_path.remove
                        ]',' '||chr(10)),' '||chr(10));
  v:=substr(v,instr(v,m)+length(m));
  v:=substr(v,1,instr(v,'notfound.')-1);
  :cur_path:=v;
end;
/
set scan off;
ho (rm _cur_path.remove 2>&1  | echo .)
ho (del _cur_path.remove 2>&1 | echo .)
col cur_path new_val cur_path noprint;
select :cur_path cur_path from dual;
set scan on;
set termout on;
 
prompt Current path: &cur_path

I used here the reading file content into variable, that I already showed in the “SQL*Plus tips. #1″.
UPDATE: I’ve replaced this script with a cross platform version.

Also I did it with SED and rtrim+ltrim, because 1) I have sed even on windows; and 2) I’m too lazy to write big PL/SQL script that will support 9i-12c, i.e. without regexp_substr/regexp_replace, etc.
But of course you can rewrite it without depending on sed, if you use windows without cygwin.

PS. Note that “host pwd” returns only directory where SQL*Plus was started, but not executed script directory.

Download latest version

Categories: Development

Oracle Priority Support Infogram for 25-JUN-2015

Oracle Infogram - Thu, 2015-06-25 14:36

RDBMS



Solaris


Ops Center

From the Ops Centerblog: Ops Center 12.3 is Released

Java

Building Simple Java EE REST Service Using Oracle JDeveloper 12c, from Oracle Partner Hub: ISV Migration Center Team.

Asynchronous Processing, from The Java Source.

JDeveloper and ADF

The 10 Most Recently Created Notes for JDeveloper/ADF as of 22 June 2015, from Proactive Support - Java Development using Oracle Tools.

, from WebLogic Partner Community EMEA.

And from the same source:




NetBeans


JavaScript


Web Computing

HTTP/2 and Server Push, from The Aquarium.

Security


EBS

From the Oracle E-Business Suite Support blog:




From the Oracle E-Business Suite Technology blog:


…And Finally


I’m (finally!) back on iPhone after an unpleasant jaunt into Android telephony and found a weather app I love. It makes use of the built in barometer on iPhone combined with user reporting, radar, etc. to give really accurate predictions on precipitation. I tested it the other day with an approaching thunderstorm and it was accurate on the arrival and intensity of the rain to within a few minutes: Dark Sky update looks to 'revolutionize weather forecasting’ by tapping iPhone sensors, from The Verge.

Hive (HiveQL) SQL for Hadoop Big Data

Kubilay Çilkara - Thu, 2015-06-25 14:30


In this  post I will share my experience with an Apache Hadoop component called Hive which enables you to do SQL on an Apache Hadoop Big Data cluster.

Being a great fun of SQL and relational databases, this was my opportunity to set up a mechanism where I could transfer some (a lot)  data from a relational database into Hadoop and query it with SQL. Not a very difficult thing to do these days, actually is very easy with Apache Hive!

Having access to a Hadoop cluster which has the Hive module installed on, is all you need. You can provision a Hadoop cluster yourself by downloading and installing it in pseudo mode on your own PC. Or you can run one in the cloud with Amazon AWS EMR in a pay-as-you-go fashion.

There are many ways of doing this, just Google it and you will be surprised how easy it is. It is easier than it sounds. Next find links for installing it on your own PC (Linux).  Just download and install Apache Hadoop and Hive from Apache Hadoop Downloads

You will need to download and install 3 things from the above link.

  • Hadoop (HDFS and Big Data Framework, the cluster)
  • Hive (data warehouse module)
  • Sqoop (data importer)
You will also need to put the connector of the database (Oracle, MySQL...) you want to extract data from in the */lib folder in your Sqoop installation. For example the MySQL JDBC connector can be downloaded from hereDon't expect loads of tinkering installing Apache Hadoop and Hive or Sqoop, just unzipping binary extracts and few line changes on some config files in directories, that's all. Is not a big deal, and is Free. There are tones of tutorials on internet on this, here is one I used from another blogger bogotobogo.


What is Hive?

Hive is Big Data SQL, the Data Warehouse in Hadoop. You can create tables, indexes, partition tables, use external tables, Views like in a relational database Data Warehouse. You can run SQL to do joins and to query the Hive tables in parallel using the MapReduce framework. It is actually quite fun to see your SQL queries translating to MapReduce jobs and run in parallel like parallel SQL queries we do on Oracle EE Data Warehouses and other databases. :0) The syntax looks very much like MySQL's SQL syntax.

Hive is NOT an OLTP transactional database, does not have transactions of INSERT, UPDATE, DELETE like in OLTP and doesn't conform to ANSI SQL and ACID properties of transactions.


Direct insert into Hive with Apache Sqoop:

After you have installed Hadoop and have hive setup and are able to login to it, you can use Sqoop - the data importer of Hadoop - like in the following command and directly import a table from MySQL via JDBC into Hive using MapReduce.

$  sqoop import -connect jdbc:mysql://mydatbasename -username kubilay -P -table mytablename --hive-import --hive-drop-import-delims --hive-database dbadb --num-mappers 16 --split-by id

Sqoop import options explained:
  •  -P will ask for the password
  • --hive-import which makes Sqoop to import data straight into hive table which it creates for you
  • --hive-drop-import-delims Drops \n\r, and \01 from string fields when importing to Hive. 
  • --hive-database tells it which database in Hive to import it to, otherwise it goes to the default database. 
  • --num-mappers number of parallel maps to run, like parallel processes / threads in SQL
  • --split-by  Column of the table used to split work units, like in partitioning key in database partitioning. 
The above command will import any MySQL table you give in place of mytablename into Hive using MapReduce from a MySQL database you specify.

Once you import the table then you can login to hive and run SQL to it like in any relational database. You can login to Hive in a properly configured system just by calling hive from command line like this:

$ hive
hive> 


More Commands to list jobs:

Couple of other commands I found useful when I was experimenting with this:

List running Hadoop jobs

hadoop job -list

Kill running Hadoop jobs

hadoop job -kill job_1234567891011_1234

List particular table directories in HDFS

hadoop fs -ls mytablename


More resources & Links



Categories: DBA Blogs

Groovy Time! How to use XML dateTime and duration in BPM 12c

Jan Kettenis - Wed, 2015-06-24 14:27
In this article I show some examples of handling XML dateTime and durations in Groovy in the context of a Oracle BPM 12c application.

Working with dates and durations in Java has always been painful. Mainly because date and time is a complex thing, with different formats and time zones and all, but I sometimes wonder if it has not been made overly complex. Anyway. Working with XML dates is even more complex because the limited support by XPath functions. Too bad because in BPM applications that work with dates this has to be done very often, and as a result I very often see the need to create all kinds of custom XPath functions to mitigate that.

This issue of complexity is no different for Groovy scripting in Oracle BPM 12c. And let handling of dates be a typical use case for using Groovy scripting because of this limited support by XPath. Therefore, to get you started (and help myself some next time) I would like to share a couple of Groovy code snippets for working with XML dates and durations that may be useful. These example are based on working with the XML dateTime type, and do not handle with the complexity of time zones and different formats. In my practice this is 99% of the use cases that I see.

In my opinion you still should limit using Groovy to handle dates and to the minimum, and rather use custom XPath functions, or create a Java library which you can can import in Groovy. But when you have to, this just might come in handy.

Instantiate an XML DateIf you have an XML element of type dateTime, you use an XmlCalender object. An XmlCalender object with the current time can instantiated as shown below:

Date now = new Date()
GregorianCalendar gregorianNow = new GregorianCalendar()
gregorianNow.setTime(now)
XmlCalendar xmlDate = XmlCalendarFactory.create(gregorianNow)


Instantiate a Duration and Add it to the DateTo instantiate a duration you use an XmlDuration object. In the code below a duration of one day is added to the date:

XmlDuration xmlDuration = new XmlDuration("P1D")
xmlDate.add(xmlDuration)


The string to provide is of type ISO duration.

The imports to use can also be a pain to find. That actually took me the most time of all, but that can just be me. The ones needed for the above are shown in the following picture (you can get to it by using clicking on Select Imports on the top-right corner of the Groovy script.

Pages

Subscribe to Oracle FAQ aggregator