Skip navigation.

Development

Searching and installing Linux packages

Arun Bavera - Tue, 2014-08-12 11:12

yum search vnc

Loaded plugins: security

public_ol6_UEKR3_latest | 1.2 kB 00:00

public_ol6_UEKR3_latest/primary | 7.7 MB 00:01

public_ol6_UEKR3_latest 216/216

public_ol6_latest | 1.4 kB 00:00

public_ol6_latest/primary | 41 MB 00:03

public_ol6_latest 25873/25873

=============================================================================== N/S Matched: vnc ================================================================================

gtk-vnc.i686 : A GTK widget for VNC clients

gtk-vnc.x86_64 : A GTK widget for VNC clients

gtk-vnc-devel.i686 : Libraries, includes, etc. to compile with the gtk-vnc library

gtk-vnc-devel.x86_64 : Libraries, includes, etc. to compile with the gtk-vnc library

gtk-vnc-python.x86_64 : Python bindings for the gtk-vnc library

libvncserver.i686 : Library to make writing a vnc server easy

libvncserver.x86_64 : Library to make writing a vnc server easy

libvncserver-devel.i686 : Development files for libvncserver

libvncserver-devel.x86_64 : Development files for libvncserver

tigervnc.x86_64 : A TigerVNC remote display system

tigervnc-server.x86_64 : A TigerVNC server

tigervnc-server-applet.noarch : Java TigerVNC viewer applet for TigerVNC server

tigervnc-server-module.x86_64 : TigerVNC module to Xorg

tsclient.x86_64 : Client for VNC and Windows Terminal Server

vinagre.x86_64 : VNC client for GNOME

xorg-x11-server-source.noarch : Xserver source code required to build VNC server (Xvnc)

Name and summary matches only, use "search all" for everything.

yum install tigervnc-server.x86_64

Loaded plugins: security

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package tigervnc-server.x86_64 0:1.1.0-8.el6_5 will be installed

--> Processing Dependency: xorg-x11-fonts-misc for package: tigervnc-server-1.1.0-8.el6_5.x86_64

--> Running transaction check

---> Package xorg-x11-fonts-misc.noarch 0:7.2-9.1.el6 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================================================

Package Arch Version Repository Size

=================================================================================================================================================================================

Installing:

tigervnc-server x86_64 1.1.0-8.el6_5 public_ol6_latest 1.1 M

Installing for dependencies:

xorg-x11-fonts-misc noarch 7.2-9.1.el6 public_ol6_latest 5.8 M

Transaction Summary

=================================================================================================================================================================================

Install 2 Package(s)

Total download size: 6.9 M

Installed size: 9.7 M

Is this ok [y/N]: y

Downloading Packages:

(1/2): tigervnc-server-1.1.0-8.el6_5.x86_64.rpm | 1.1 MB 00:00

(2/2): xorg-x11-fonts-misc-7.2-9.1.el6.noarch.rpm | 5.8 MB 00:01

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Total 2.5 MB/s | 6.9 MB 00:02

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

Warning: RPMDB altered outside of yum.

Installing : xorg-x11-fonts-misc-7.2-9.1.el6.noarch 1/2

Installing : tigervnc-server-1.1.0-8.el6_5.x86_64 2/2

Verifying : xorg-x11-fonts-misc-7.2-9.1.el6.noarch 1/2

Verifying : tigervnc-server-1.1.0-8.el6_5.x86_64 2/2

Installed:

tigervnc-server.x86_64 0:1.1.0-8.el6_5

Dependency Installed:

xorg-x11-fonts-misc.noarch 0:7.2-9.1.el6

Complete!

Categories: Development

Test your WebLogic 12.1.3 enviroment with Robot

Edwin Biemond - Sun, 2014-08-10 11:42
Robot Framework is a generic test automation framework which has an easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach. This means we can write our tests in readable and understandable text. If we combine this with the REST Management interface of WebLogic 12.1.3 we are able to test every detail of a WebLogic domain configuration and when we combine this

Required Field Validation in Oracle MAF

Shay Shmeltzer - Fri, 2014-08-08 16:38

A short entry to explain how to do field validation in Oracle MAF. As an example let's suppose you want a field to have a value before someone clicks to do an operation.

To do that you can set the field's attribute for required and "show required" like this:

  <amx:inputText label="label1" id="it1" required="true" showRequired="true"/>

 Now if you run your page, leave the field empty and click a button that navigates to another page, you'll notice that there was no indication of an error. This is because you didn't tell the AMX page to actually do a validation. 

 To add validation you use an amx:validationGroup tag that will surround the fields you want to validate.

For example:

     <amx:validationGroup id="vg1">

    <amx:inputText label="label1" id="it1" required="true" showRequired="true"/>

    </amx:validationGroup>

Then you can add a amx:validateOperation tag to the button that does navigation and tell it to validate the group you defined before (vg1 in our example).

       <amx:commandButton id="cb2" text="go" action="gothere">

        <amx:validationBehavior id="vb1" group="vg1"/>

      </amx:commandButton>

Now when you run the page and try to navigate you'll get your validation error.

Categories: Development

Create with WLST a SOA Suite, Service Bus 12.1.3 Domain

Edwin Biemond - Thu, 2014-08-07 13:14
When you want to create a 12.1.3 SOA Suite, Service Bus Domain, you have to use the WebLogic config.sh utility.  The 12.1.3 config utility is a big improvement when you compare this to WebLogic 11g. With this I can create some complex cluster configuration without any after configuration. But if you want to automate the domain creation and use it in your own (provisioning) tool/script then you

Set Focus on Added Row Tabular

Denes Kubicek - Tue, 2014-08-05 09:19
This sounds quite simple:

"I would like to set focus on the new added row in my tabular form."

In the following example, you can se how to acomplish that.

Categories: Development

Cascading Shuttle - Keep Selected Values

Denes Kubicek - Tue, 2014-08-05 06:14
There are probably a several good ways to solve the following problem:

    1. you have a simple select list which cascades a shuttle element
    2. you select something and the shuttle is populated
    3. after that, you pick a value in the shuttle and move it to the rigt side
    4. then you change the select list again
    5. the values you selected in your shuttle are now lost


This example shows how this can be done. This question apears in the APEX Forum from time to time and I decided to provide one simple solution for it.

Categories: Development

Accessing remote databases from Oracle MAF with the TopLink/EclipseLink REST CRUD Services

Shay Shmeltzer - Mon, 2014-08-04 15:17

In the last post I showed you how simple it is to expose CRUD REST operations on your database with TopLink/EclipseLink.

The next logical step is to then consume those with Oracle MAF to build a mobile application.

This is quite simple with the REST data control. All you need to do is just map the right URLs and create the operation.

Here is a quick demo:

One trick I show in the demo is how to delay the call to a REST service until the user actually provides a value to a parameter. A common issue people have when they have the parameter form and the results on the same page. The solution is easy using the refresh condition of the executables of the page and using the "ne null" check on the parameter value. 

Categories: Development

REST enable your Database for CRUD with TopLink/EclipseLink and JDeveloper

Shay Shmeltzer - Fri, 2014-08-01 17:10

It seems that REST interfaces are all the rage now for accessing your backend data, this is especially true in the world of mobile development. In this blog I'm going to show you how easy it is to provide a complete REST interface for your database by leveraging TopLink/EclipseLink and JDeveloper.

This relies on a capability that is available in TopLink 12c where every JPA entity that you have created can be RESTified with a simple servlet that TopLink provides.

All you need to do is locate the file toplink-dataservices-web.jar on your machine (this is included in the JDeveloper install so you can just search that directory) and then package your project as a WAR.

At that point you'll be able to get a complete CRUD set of operation for this entity.

In the video below I'm to retrieving departments by their id using a URL like this:

http://127.0.0.1:7101/TLServices-Project1-context-root/persistence/v1.0/out/entity/Departments/30

(out - name of my persistence unit. Departments - name of my entity) 

A complete list of all the REST URL syntax is here part of the TopLink documentation on this feature.:

http://docs.oracle.com/middleware/1213/toplink/solutions/restful_jpa.htm#CHDEGJIG

Check out how easy the process is in this video (using MySQL database):

Here are some additional URL samples for getting other types of queries:

Get all the Employees -  http://127.0.0.1:7101/TLServices/persistence/v1.0/out/query/Employees.findAll

Get all the Employees in department 50 - http://127.0.0.1:7101/TLServices/persistence/v1.0/out/entity/Departments/50/employeesList

Executing a specific named query (@NamedQuery(name = "Employees.findByName", query = "select o from Employees o where o.first_name like :name order by o.last_name"))  -http://127.0.0.1:7101/TLServices/persistence/v1.0/out/query/Employees.findByName;name=John

Categories: Development

jQuery - loop through a Tabular Form

Denes Kubicek - Thu, 2014-07-31 07:48
This question is one of the frequently asked questions - "How do I loop through a tabular form using a dynamic action?". This example shows how to loop through a tabular form and set the values for each row to what ever you want. Using apex_application.g_fxx array is not an option for onload processes or dynamic actions. It can only be used in an on submit process. Using jQuery in a simple loop it is possible to read / set any of the values in any column. Try it out.

Categories: Development

Test your Application with the WebLogic Maven plugin

Edwin Biemond - Thu, 2014-07-31 05:47
In this blogpost I will show you how easy it is to add some unit tests to your application when you use Maven together with the 12.1.3 Oracle software ( like WebLogic , JDeveloper or Eclipse OEPE). To demonstrate this, I will create a RESTful Person Service in JDeveloper 12.1.3 which will use the Maven project layout. We will do the following: Create a Project and Application based on a Maven

OTN APEX Forum again

Denes Kubicek - Thu, 2014-07-31 04:02
The OTN Forum is not available (again). This usefull but constantly changing forum now gives me the following message:



I am not sure why they are using jive for that. Maybe APEX would be a better solution.
Categories: Development

Silence

Greg Pavlik - Sat, 2014-07-26 10:26
Silence. Sometimes sought after, but in reality almost certainly feared - the absence of not just sound but voice. Silence is often associated with divine encounter - the neptic tradition of the Philokalia comes to mind - but also and perhaps more accurately with abandonment, divine or otherwise. I recently read Shusaku Endo's Silence, a remarkable work, dwelling on the theme of abandonment in the context of the extirpation of Kakure Kirishitan communities in Tokagawa Japan. Many resilient families survived and eventually came out of hiding in the liberalization in the mid-19th century, but the persecutions were terrible. Their story is deeply moving (sufficiently so that over time I find myself drawn to devotion to the image of Maria-Kannon). Endo's novel was not without controversy but remains one of the great literary accomplishments of the 20th century.

In fact, the reason for this post is a kind of double entendre on silence: the relative silence in literate western circles with respect to Japanese literature of the past century. Over the last month, I realized that virtually no one I had spoken with had read a single Japanese novel. Yet, like Russia of the 19th century, Japan produced a concentration of great writers and great novelists in the last 20th century that is set apart: the forces of of profound national changes (and defeat) created the crucible of great art. That art carries the distinctive aesthetic sense of Japan - a kind of openness of form, but is necessarily the carrier of universal, humanistic themes.

Endo is a writer in the post war period - the so-called third generation, and in my view the last of the wave of great Japanese literature. Read him. But don't stop - perhaps don't start - there. The early 20th century work of Natsume Soseki are a product of the Meiji period. In my view, Soseki is not only a father of Japenese literature but one of the greatest figures of world literature taken as a whole - I am a Cat remains one of my very favorite novels. Two troubling post-war novels by Yukio Mishima merit attention - Confessions of a Mask and the Sailor Who Fell From Grace with the Sea, both I would characterize broadly as existential masterpieces. The topic of identity in the face of westernization is also a moving theme in Osamu Dazai's No Longer Human. I hardly mean this as a complete survey - something in any case I am not qualified to provide -just a pointer toward something broader and important.

My encounter with contemporary Japanese literature - albeit limited - has been less impactful (I want to like Haruki Murakami in the same way I want to like Victor Pelevin, but both make me think of the distorted echo of something far better). And again like Russia, it is difficult to know what to make of Japan today - where its future will lead, whether it will see a cultural resurgence or decline. It is certain that its roots are deep and I hope she finds a way to draw on them and to flourish.


How to find out session info about session that comes from remote database through dblink

XTended Oracle SQL - Thu, 2014-07-24 19:28
.syntaxhighlighter { width: 1800px; overflow-x: auto !important;}

It is well known thing and you can even find it on MOS, but I have a little more simple script for it, so I want to show little example.

First of all we need to start script on local database:

SQL>                                                                                                                                                                   
SQL> @transactions/global.sql
Enter filters(empty for any)...
Sid           :
Globalid mask :
Remote_db mask:

 INST_ID  SID    SERIAL# USERNAME REMOTE_DB REMOTE_DBID TRANS_ID         DIRECTION   GLOBALID                                           EVENT                      
-------- ---- ---------- -------- --------- ----------- ---------------- ----------- -------------------------------------------------- ---------------------------
       1  275       4469 XTENDER  BAIKAL     1742630060 8.20.7119        FROM REMOTE 4241494B414C2E63616336656437362E382E32302E37313139 SQL*Net message from client
                                                                                                                                                                  

Then we need to copy GLOBALID of interested session and run script on database that shown in column REMOTE_DBID, but with specifieng GLOBALID:

SQL>                                                                                                                                                                                                 
SQL> conn sys/syspass@baikal as sysdba
Connected.

======================================================================
=======  Connected to  SYS@BAIKAL(baikal)(BAIKAL)
=======  SID           203
=======  SERIAL#       38399
=======  SPID          6536
=======  DB_VERSION    11.2.0.4.0
======================================================================

SQL> @transactions/global.sql
Enter filters(empty for any)...
Sid           :
Globalid mask : 4241494B414C2E63616336656437362E382E32302E37313139
Remote_db mask:

INST_ID   SID    SERIAL# USERNAME  REMOTE_DB  REMOTE_DBID TRANS_ID   DIRECTION   GLOBALID                                            STATE                     
------- ----- ---------- --------- ---------- ----------- ---------- ----------- --------------------------------------------------  --------------------------
      1     9      39637 XTENDER   BAIKAL      1742630060 8.20.7119  TO REMOTE   4241494B414C2E63616336656437362E382E32302E37313139  [ORACLE COORDINATED]ACTIVE

It’s quite simple and fast.

Categories: Development

Standalone sqlplus script for plans comparing

XTended Oracle SQL - Thu, 2014-07-24 18:00

I have a couple scripts for plans comparing:

1. https://github.com/xtender/xt_scripts/blob/master/diff_plans.sql
2. http://github.com/xtender/xt_scripts/blob/master/plans/diff_plans_active.sql

But they have dependencies on other scripts, so I decided to create a standalone script for more convenient use without the need to download other scripts and to set up the sql*plus environment.
I’ve tested it already with firefox, so you can try it now: http://github.com/xtender/xt_scripts/blob/master/plans/diff_plans_active_standalone.sql

Some screenshots:
diff_plans.sql:
diff_plans

plans_active.sql:
plans_active

Usage:
1. plans_active:

SQL> @plans_active 0ws7ahf1d78qa 

2. diff_plans:

SQL> @diff_plans 0ws7ahf1d78qa 
 *** Diff plans by sql_id. Version with package XT_PLANS. 
Usage: @plans/diff_plans2 sqlid [+awr] [-v$sql] 

P_AWR           P_VSQL 
--------------- --------------- 
false           true 

Strictly speaking, we can do it sometimes easier: it’s quite simple to compare plans without first column “ID”, so we can simply compare “select .. from v$sql_plan/v$sql_plan_statistics_all/v$sql_plan_monitor” output with any comparing tool.

Categories: Development

Bug with xmltable, xmlnamespaces and xquery_string specified using bind variable

XTended Oracle SQL - Thu, 2014-07-24 12:54

Today I was asked about strange problem: xmltable does not return data, if xquery specified by bind variable and xml data has xmlnamespaces:

SQL> var x_path varchar2(100);
SQL> var x_xml  varchar2(4000);
SQL> col x format a100;
SQL> begin
  2      :x_path:='/table/tr/td';
  3      :x_xml :=q'[
  4                  <table xmlns="http://www.w3.org/tr/html4/">
  5                    <tr>
  6                      <td>apples</td>
  7                      <td>bananas</td>
  8                    </tr>
  9                  </table>
 10                  ]';
 11  end;
 12  /

PL/SQL procedure successfully completed.

SQL> select
  2        i, x
  3   from xmltable( xmlnamespaces(default 'http://www.w3.org/tr/html4/'),
  4                  :x_path -- bind variable
  5                  --'/table/tr/td' -- same value as in the variable "X_PATH"
  6                  passing xmltype(:x_xml)
  7                  columns i    for ordinality,
  8                          x    xmltype path '.'
  9                );

no rows selected

But if we comment bind variable and comment out literal x_query ‘/table/tr/td’, query will return data:

SQL> select
  2        i, x
  3   from xmltable( xmlnamespaces(default 'http://www.w3.org/tr/html4/'),
  4                  --:x_path -- bind variable
  5                  '/table/tr/td' -- same value as in the variable "X_PATH"
  6                  passing xmltype(:x_xml)
  7                  columns i    for ordinality,
  8                          x    xmltype path '.'
  9                );

         I X
---------- -------------------------------------------------------------------
         1 <td xmlns="http://www.w3.org/tr/html4/">apples</td>
         2 <td xmlns="http://www.w3.org/tr/html4/">bananas</td>

2 rows selected.

The only workaround I found is the specifying any namespace in the x_query – ‘/*:table/*:tr/*:td’

SQL> exec :x_path:='/*:table/*:tr/*:td'

PL/SQL procedure successfully completed.

SQL> select
  2        i, x
  3   from xmltable( xmlnamespaces(default 'http://www.w3.org/tr/html4/'),
  4                  :x_path -- bind variable
  5                  passing xmltype(:x_xml)
  6                  columns i    for ordinality,
  7                          x    xmltype path '.'
  8                );

         I X
---------- -------------------------------------------------------------------
         1 <td xmlns="http://www.w3.org/tr/html4/">apples</td>
         2 <td xmlns="http://www.w3.org/tr/html4/">bananas</td>

2 rows selected.

It’s quite ugly solution, but I’m not sure whether there is another solution…

Categories: Development

Spark: A Discussion

Greg Pavlik - Wed, 2014-07-23 08:36
A great presentation, worth watching in its entirety.

With apologies to my Hadoop friends but this is good for you too.

REGEXP_LIKE: strange unspecified value in parameter “modifier”

XTended Oracle SQL - Tue, 2014-07-22 15:05

Today I noticed strange thing in predicate section of execution plan for simple query with regexp_like, where 3rd parameter “MODIFIER” was not specified:

SQL> select * from dual where regexp_like(dummy,'.');

D
-
X

SQL> select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------
SQL_ID  97xuqf9cmjsta, child number 0
-------------------------------------
select * from dual where regexp_like(dummy,'.')

Plan hash value: 272002086

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |       |       |     2 (100)|          |
|*  1 |  TABLE ACCESS FULL| DUAL |     1 |     2 |     2   (0)| 00:00:01 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter( REGEXP_LIKE ("DUMMY",'.',HEXTORAW('F07FD85CFF0700006A1116
              45010000000000000000000000FC12164501000000000000000000000000000000000000
              0010000000000000001880D85CFF07000002000000000000000000000081000000') ))


20 rows selected.

It is particularly interesting that the values in HEXTORAW() are always different for different first parameters:

SQL> select * from dual where regexp_like(dummy,'x');
...
   1 - filter( REGEXP_LIKE ("DUMMY",'x',HEXTORAW('3895D330FF0700006A1116
              45010000000000000000000000FC12164501000000000000000000000000000000000000
              0011000000000000006895D330FF07000002000000000000000000000081000000') ))
SQL> select * from dual where regexp_like(dummy,'y');
...
   1 - filter( REGEXP_LIKE ("DUMMY",'y',HEXTORAW('00DA3C3FFF0700006A1116
              45010000000000000000000000FC12164501000000000000000000000000000000000000
              00110000000000000030DA3C3FFF07000002000000000000000000000081000000') ))
SQL> select * from dual where regexp_like(dummy||'','x')
...
   1 - filter( REGEXP_LIKE ("DUMMY"||'','x',HEXTORAW('70964F2FFF0700006A
              111645010000000000000000000000FC1216450100000000000000000000000000000000
              0000001100000000000000A0964F2FFF07000002000000000000000000000081000000')
               ))

I don’t know, what does it mean, but it looks like garbage from memory.
When I noticed this, I decided to check how regexp_like will work in function-based indexes:

SQL> create table xtest as
  2    select dummy||level as str
  3    from dual
  4    connect by level<=30;

Table created.

SQL> select * from xtest where case when regexp_like(str,'1') then 1 end = 1;
...
12 rows selected.

SQL> select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------
SQL_ID  7ztp0k8c1zn2h, child number 0
-------------------------------------
select * from xtest where case when regexp_like(str,'1') then 1 end = 1

Plan hash value: 4207139086

---------------------------------------------------------------------------
| Id  | Operation         | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |       |       |       |     3 (100)|          |
|*  1 |  TABLE ACCESS FULL| XTEST |    12 |   264 |     3   (0)| 00:00:01 |
---------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(CASE  WHEN  REGEXP_LIKE
              ("STR",'1',HEXTORAW('68F9CB32FF0700006A111645010000000000000000000000FC1
              216450100000000000000000000000000000000000000110000000000000098F9CB32FF0
              7000002000000000000000000000081000000') ) THEN 1 END =1)

SQL> create index xtest_fbi on xtest(case when regexp_like(str,'1') then 1 end);

Index created.

SQL> select * from xtest where case when regexp_like(str,'1') then 1 end = 1;
...
12 rows selected.

SQL> select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------
SQL_ID  7ztp0k8c1zn2h, child number 0
-------------------------------------
select * from xtest where case when regexp_like(str,'1') then 1 end = 1

Plan hash value: 1479471124

-----------------------------------------------------------------------------------------
| Id  | Operation                   | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |           |       |       |     2 (100)|          |
|   1 |  TABLE ACCESS BY INDEX ROWID| XTEST     |    12 |   300 |     2   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | XTEST_FBI |    12 |       |     1   (0)| 00:00:01 |
-----------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("XTEST"."SYS_NC00002$"=1)

SQL> select column_expression from user_ind_expressions e where e.index_name='XTEST_FBI';

COLUMN_EXPRESSION
-----------------------------------------------------------------------------------------
CASE  WHEN  REGEXP_LIKE ("STR",'1') THEN 1 END

As you can see it works fine, although the predicate from first execution plan differs from the FBI expression.
Then I dumped 10053 trace and noticed that the HEXTORAW(…) function appeared in “Explain Plan Dump” only, so it looks just like plan output bug.

Categories: Development

Exactly Wrong

Greg Pavlik - Mon, 2014-07-21 08:58
I normally avoid anything that smacks of a competitive discussion on what I consider to be a space for personal reflection. So while I want to disclose the fact that I am not disinterested in the points I am making from a professional standpoint, my main interest is to frame some architecture points that I think are extremely important for the maturation and success of the Hadoop ecosystem.

A few weeks back, Mike Olson of Cloudera spoke at Spark Summit on how Spark relates to the future of Hadoop. The presentation can be found here:

http://youtu.be/8kcdwnbHnJo

In particular I want to draw attention to the statement made at 1:45 in the presentation that describes Spark as the "natural successor to MapReduce" - it becomes clear very quickly that what Olson is talking about is batch processing. This is fascinating as everyone I've talked to immediately points out one obvious thing: Spark isn't a general purpose batch processing framework - that is not its design center. The whole point of Spark is to enable fast data access and interactivity.
 
The guys that clearly "get" Spark - unsurprisingly - are DataBricks. In talking with Ion and company, it's clear they understand the use cases where Spark shines - data scientist driven data exploration and algorithmic development, machine learning, etc. - things that take advantage of the memory mapping capabilities and speed of the framework. And they have offered an online service that allows users to rapidly extract value from cloud friendly datasets, which is smart.

Cloudera's idea of pushing SQL, Pig and other frameworks on to Spark is actually a step backwards - it is a proposal to recreate all the problems of MapReduce 1: it fails to understand the power of refactoring resource management away from the compute model. Spark would have to reinvent and mature models for multi-tenancy, resource managemnet, scheduling, security, scaleout, etc that are frankly already there today for Hadoop 2 with YARN.

The announcement of an intent to lead an implementation of Hive on Spark got some attention. This was something that I looked at carefully with my colleagues almost 2 years ago, so I'd like to make a few observations on why we didn't take this path then.

The first was maturity, in terms of the Spark implementation, of Hive itself, and Shark. Candidly, we knew Hive itself worked at scale but needed significant enhancement and refactoring for both new features on the SQL front and to work at interactive speeds. And we wanted to do all this in a way that did not compromise Hive's ability to work at scale - for real big data problems. So we focused on the mainstream of Hive and the development of a Dryad like runtime for optimal execution of operators in physical plans for SQL in a way that meshed deeply with YARN. That model took the learnings of the database community and scale out big data solutions and built on them "from the inside out", so to speak.

Anyone who has been tracking Hadoop for, oh, the last 2-3 years will understand intuitively the right architectural approach needs to be based on YARN. What I mean is that the query execution must - at the query task level - be composed of tasks that are administered directly by YARN. This is absolutely critical for multi-workload systems (this is one reason why a bolt on MPP solution is a mistake for Hadoop - it is at best a tactical model while the system evolves).  This is why we are working with the community on Tez, a low level framework for enabling YARN native domain specific execution engines. For Hive-on-Tez, Hive is the engine and Tez provides the YARN level integration for resource negotiation and coorindation for DAG execution: a DAG of native operators analogous the the execution model found in the MPP world (when people compare Tez and Spark, they are fundamentally confused - Spark could be run on Tez for example for a much deeper integration with Hadoop 2 for example). This model allows the full range of use cases from interactive to massive batch to be administered in a deeply integrated, YARN native way.

Spark will undoubtedly mature into a great tool for what it is designed for: in memory, interactive scenarios - generally script driven - and likely grow to subsume new use cases we aren't anticipating today. It is, however, exactly the wrong choice for scale out big data batch processing in anything like the near term; worse still, returning to a monolithic general purpose compute framework for all Hadoop models would be a huge regression and is a disastrously bad idea.

OTN APEX Forum Link

Denes Kubicek - Mon, 2014-07-21 00:33
Oracle again changed the layout of the forum. For me, the old link didn't work any more. In case you have problems finding it, here is the new link:

https://community.oracle.com/community/database/developer-tools/application_express

If you go to the forum and search for example for "APEX" or "Application Exp", you will see no results. Typing in "Application Ex" will find "Application Express".



Each of the found links will have a funny description saying:

"An error occurred processing your request. If this problem persists, please contact the webmaster or administrator of this site."



:) So, it seems there are now even more bugs than before.

Probably, the intention to change the forum wasn't bad. However, once you manage to open it you will see a lot of information you don't need (or at least not all of the time). The real content is somewhere underneath and needs scrolling like in Facebook (oh, how I hate that site). And the worst thing is that you can see only ten threads per page - if you want to see more then click and scroll again. For those interested in helping others this is making things much more complicated.



One positive thing though. :) My name suddenly appears in the top list of the participants in the forum. The list isn't reduced to the top five but it now shows the top six. Top six is obviously the new top five. ;)

Categories: Development

Dependent Rational Animals

Greg Pavlik - Sun, 2014-07-20 16:32
I wanted to briefly comment on Alisdair MacIntyre's lectures collected as "Dependent Rational Animals", but let me precede that with a couple of comments for context: first, as I alluded in my last post referencing Levinas, it is my view that the the ethics demands a certain primacy in any healthy conception of life and society; second, in the area of ethics, Macintyre's After Virtue is the book that has had perhaps the biggest impact on my own thinking.

One of the criticisms of MacIntyre is that his critique of rational ethics is, on the one hand, devastating; on the other hand, his positive case for working out a defense of his own position - a revivification of social ethics in the Aristotelian-Thomist tradition(s) was somewhat pro forma. I think this is legitimate in so far as it relates to After Virtue itself (I believe I have read the latest edition - 3 - most recently), though I am not enough of a MacIntyre expert to offer a defensible critique of his work overall.

I do, however, want to draw attention to Dependent Rational Animals specifically in this light. Here MacIntyre begins with is the position of human as animal - as a kind of naturalist starting point for developing another pass at the importance of the tradition of the virtues. What is most remarkable is that in the process of exploring the implications of our "animality" MacIntyre manages to subvert yet another trajectory of twentieth century philosophy, this time as it relates to the primacy of linguistics. The net effect is to restore philosophical discourse back toward the reality of the human condition in the context of the broader evolutionary context of life on earth without - and this I must say is the most amazing part of this book - resorting to fables-masked-as-science (evolutionary psychology).