Skip navigation.

Christopher Jones

Syndicate content
Notes on the Scripting Languages and Oracle Database. I cover Node.js, PHP, Python, Ruby, Perl, Apache, Linux
Updated: 10 hours 20 min ago

Node-oracledb: Avoiding "ORA-01000: maximum open cursors exceeded"

Fri, 2016-02-05 05:45

Developers starting out with Node have to get to grips with the 'different' programming style of JavaScript that seems to cause methods to be called when least expected! While you are still in the initial hacking-around-with-node-oracledb phase you may sometimes encounter the error ORA-01000: maximum open cursors exceeded.

Here are some things to do about it:

  • Avoid having too many incompletely processed statements open at one time:

    • Close ResultSets before releasing the connection.

    • If cursors are opened with dbms_sql.open_cursor() in a PL/SQL block, close them before the block returns - except for REF CURSORS being passed back to node-oracledb. (And if a future node-oracledb version supports Oracle Database 12c Implicit Result Sets, these cursors should likewise not be closed in the PL/SQL block)

    • Make sure your application is handling connections and statements in the order you expect.

  • Choose the appropriate Statement Cache size. Node-oracledb has a statement cache per connection. When node-oracledb internally releases a statement it will be put into the statement cache of that connection, but its cursor will remain open. This makes statement re-execution very efficient.

    The cache size is settable with the stmtCacheSize attribute. The appropriate statement cache size you choose will depend on your knowledge of the locality of the statements, and of the resources available to the application: are statements re-executed; will they still be in the cache when they get executed; how many statements do you want to be cached? In rare cases when statements are not re-executed, or are likely not to be in the cache, you might even want to disable the cache to eliminate its management overheads.

    Incorrectly sizing the statement cache will reduce application efficiency. Luckily with Oracle 12.1, the cache can be automatically tuned using an oraaccess.xml file.

    More information on node-oracledb statement caching is here.

  • Don't forget to use bind variables otherwise each variant of the statement will have its own statement cache entry and cursor. With appropriate binding, only one entry and cursor will be needed.

  • Set the database's open_cursors parameter appropriately. This parameter specifies the maximum number of cursors that each "session" (i.e each node-oracle connection) can use. When a connection exceeds the value, the ORA-1000 error is thrown. Documentation on open_cursors is here.

    Along with a cursor per entry in the connection's statement cache, any new statements that a connection is currently executing, or ResultSets that haven't been released (in neither situation are these yet cached), will also consume a cursor. Make sure that open_cursors is large enough to accommodate the maximum open cursors any connection may have. The upper bound required is stmtCacheSize + the maximum number of executing statements in a connection.

    Remember this is all per connection. Also cache management happens when statements are internally released. The majority of your connections may use less than open_cursors cursors, but if one connection is at the limit and it then tries to execute a new statement, that connection will get ORA-1000: maximum open cursors exceeded.

node-oracledb 1.6.0 is on NPM (Node.js add-on for Oracle Database)

Sat, 2016-01-30 06:07
Node-oracledb 1.6.0, the Node.js add-on for Oracle Database, is on NPM.

In this release a comprehensive pull request by Dieter Oberkofler adds support for binding PL/SQL Collection Associative Array (Index-by) types. Strings and numbers can now be bound and passed to and from PL/SQL blocks. Dieter tells us that nowadays he only gets to code for a hobby - keep it up Dieter!

Using PL/SQL Associative Arrays can be a very efficient way of transferring database between an application and the database because it can reduce the number of 'round trips' between the two.

As an example, consider this table and PL/SQL package:

  CREATE TABLE mytab (numcol NUMBER);

  CREATE OR REPLACE PACKAGE mypkg IS
    TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
    PROCEDURE myinproc(p IN numtype);
  END;
  /

  CREATE OR REPLACE PACKAGE BODY mypkg IS
    PROCEDURE myinproc(p IN numtype) IS
    BEGIN
      FORALL i IN INDICES OF p
	INSERT INTO mytab (numcol) VALUES (p(i));
    END;
  END;
  /

With this schema, the following JavaScript will result in mytab containing five rows:

  connection.execute(
    "BEGIN mypkg.myinproc(:bv); END;",
    {
      bv: { type : oracledb.NUMBER,
	    dir: oracledb.BIND_IN,
	    val: [1, 2, 23, 4, 10]
	  }
    },
    function (err) { . . . });

There is a fuller example in examples/plsqlarray.sql and check out the documentation.

Other changes in node-oracledb 1.6 are

  • @KevinSheedy sent a GitHub Pull Request for the README to help the first time reader have the right pre-requisites and avoid the resulting pitfalls.

  • Fixed a LOB problem causing an uncaught error to be generated.

  • Removed the 'close' event that was being generated for LOB Writables Streams. The Node.js Streams doc specifies it only for Readable Streams.
  • Updated the LOB examples to show connection release.

  • Extended the OS X install section with a way to install on El Capitan that doesn't need root access for Instant Client 11.2. Thanks to @raymondfeng for pointing this out.

  • Added RPATH to the link line when building on OS X in preparation for future client.

TypeScript users will be happy to hear Richard Natal recently had a node-oracledb TypeScript type definition file added to the DefinitelyTyped project. This is not part of node-oracledb itself but Richard later mentioned he found a way it could be incorporated. Hopefully he will submit a pull request and it will make it directly to the project so it can be kept in sync.

Thanks to everyone who has worked on this release and kept the momentum going.

What's coming up for the next release? There is discussion about adding a JavaScript layer. This was kicked off by a pull request from Sagie Gur-Ari which has lead to some work by Oracle's Dan McGhan. See the discussion and let us know what you think. Having this layer could make it quicker and easier for JavaScript coders to contribute node-oracledb and do things like reduce API inconsistency, make it easier to add a promise API in future, and of course provide a place to directly add Sagie's Streaming query result suggestion that started the whole thing.

I know a few contributors have recently submitted the Oracle Contributor Agreement ready to do big and small things - every bit counts. I look forward to being able to incorporate your work.

I've heard a couple of reports that Node LTS 4.2.6 on Windows is having some issues building native add-ons. 0.10, 0.12, 5.x and 4.2.5 don't have issues. Drop me a line if you encounter a problem.

Issues and questions about node-oracledb can be posted on GitHub. We value your input to help prioritize work on the add-on. Drop us a line!

node-oracledb installation instructions are here.

Node-oracledb documentation is here.

node-oracledb 1.5.0 is on NPM (Node.js add-on for Oracle Database)

Mon, 2015-12-21 06:56
Node-oracledb 1.5.0, the Node.js add-on for Oracle Database, is on NPM.

A number of bugs have been squashed in this release.

  • We now treat Oracle Database 'Success With Info' warnings as success.

    Thanks to Francisco Trevino for his pull request. After investigating and discussing, we decided for 1.5 to pick up the straightforward fix proposed. In a future release we will revisit allowing these warnings to be caught and handled.

  • Extended rollback-on-connection-release with 11g Oracle Clients to occur for all non-query executions.

    The natural behavior of OCI is to commit when a connection is released. This is the opposite of node-oracledb, which therefore has to determine whether to rollback or not.

    When node-oracledb is linked with 11g client a heuristic is used to guess whether to rollback when a connection is released. This heuristic needed to be changed to cover more cases. The result is that there will be sometimes be some unnecessary rollbacks issued.

    The bug didn't occur node-oracledb was linked with 12c client libraries due to this code that uses a new API available in 12c to indicate whether a connection has a transaction open.

    Bottom line: use Oracle 12c client libraries if possible to get optimal behavior.

  • Updated OS X install instructions to work on El Capitan.

    The instructions now use symbolic links in /usr/local/lib for the Oracle client libraries. This removes the need to set DYLD_LIBRARY_PATH, which has some restrictions on it introduced in El Capitan.

  • Display an error and prevent connection release while database calls are in progress.

    This was a bigger transaction, that 'fixed' a number of seemingly random crashes which were occurring when applications released connections that were in fact still in use. Node-oracledb will now print an error and not release the connection, thus preventing a crash. Note that since the release fails, connection pools can max out in this scenario. If you experience the errors NJS-030, NJS-031 or NJS-032 you should fix your app so the connection release occurs after all database operations have concluded.

    The new messages are:

    "NJS-030: Connection cannot be released because Lob operations are in  progress"
    "NJS-031: Connection cannot be released because ResultSet operations are in progress"
    "NJS-032: Connection cannot be released because a database call is in progress"
    
  • Fixed an intermittent crash while selecting data from CLOB column.

    We had an incorrect buffer expansion ratio in use. This has been fixed.

  • Fixed crash when trying to set invalid values for connection properties.

    Enough said.

Work for node-oracledb 1.6 will begin. We are trying to reproduce and understand some reported LOB issues and memory growth reports. We're also looking forward to evaluating a big pull request from Dieter Oberkofler that adds PL/SQL bind support.

Issues and questions about node-oracledb can be posted on GitHub. We value your input to help prioritize work on the add-on. Drop us a line!

node-oracledb installation instructions are here.

Node-oracledb documentation is here.

PHP 7 OCI8 2.1.0 available on PECL

Fri, 2015-12-11 23:47

I've released PHP 7 OCI8 2.1 on PECL and simultaneously made a patch release OCI8 2.0.10 which is compatible with PHP 5.2 - PHP 5.6.

To install OCI8 for PHP 7 use:

pecl install oci8

This installs OCI8 2.1 which, as I'm sure you can guess, had a lot of internal changes to make it compatible with the vastly changed internals of PHP 7.

If you want to install OCI8 for PHP 5.2, 5.3, 5.4, or 5.6 use:

pecl install oci8-2.0.10

Functionality in 2.0.10 and 2.1.0 is equivalent. They both contain the fix for bug 68298, an overflow when binding 64bit numbers.

[Update: Windows DLLs have been built.] At time of writing, Windows DLLs were not yet built on PECL. If you need them, you can grab them from the full PHP Windows bundle.

Major news: PHP 7.0.0 has been released

Thu, 2015-12-03 22:58

Congratulations to the PHP community - the whole community - on the release of PHP 7.0.0. Thanks also to the Oracle staff who have worked on the internal rewrite necessary to make the OCI8 and PDO_OCI extensions work with PHP 7's completely revamped Extension API.

The Oracle Database OCI8 and PDO_OCI extensions are included in the PHP source distribution. The feature sets are unchanged.

The equivalent standalone OCI8 package compatible with PHP 7 will be released as version 2.1 on PECL soon. PDO_OCI will remain solely part of the core PHP source distribution.

For those interested in performance, Zend have put some benchmark figures here showing the significant improvements, which were a key feature of this release.

Other features are listed in the release announcement:

  • Significantly reduced memory usage
  • Abstract Syntax Tree
  • Consistent 64-bit support
  • Improved Exception hierarchy
  • Many fatal errors converted to Exceptions
  • Secure random number generator
  • Removed old and unsupported SAPIs and extensions
  • The null coalescing operator (??)
  • Return and Scalar Type Declarations
  • Anonymous Classes
  • Zero cost asserts

See the migration documentation for all the fine details.

node-oracledb 1.4.0 supports Node 0.10, 0.12, 4.2 and 5 (Node.js add-on for Oracle Database)

Mon, 2015-11-16 23:35

Version 1.4 of node-oracledb, the add-on for Node.js that powers high performance Oracle Database applications, is available on NPM.

Since the recent releases of Node 4 LTS and Node 5, after the reconciliation and merge of the io.js and Node.js forks, there has been strong interest in a compatible node-oracledb driver. This is now possible. Node-oracledb 1.4 works with Node.js 0.10, 0.12, 4.2, and 5.0. Thanks to everyone for their perseverance.

The code change making this possible was a patch contributed by Richard Natal bumping the dependency on NAN from v1 to v2. Note: a compiler with support for C++11 is required to build with Node 4.2 and 5. (Oracle Linux 6 users will need to move to Oracle Linux 7 or install a newer compiler, such as from the Software Collection Library for Oracle Linux).

Other changes in this release are:

  • Fixed a statement cursor leak occuring when statements failed.

  • Fixed a crash accessing Pool properties on Windows.

  • A new testWindows target to help run the tests on Windows was added to package.json. Instructions on how to run tests are in test/README.md.

  • Fixed compilation warnings seen on some platforms with newer compilers.

Issues and questions about node-oracledb can be posted on GitHub.

node-oracledb installation instructions are here.

node-oracledb documentation is here.

Application Development with Node.js, Python, PHP, R, C, and C++ at OOW

Thu, 2015-10-22 00:18
The huge Oracle OpenWorld conference in San Francisco starts in a few days. We've put together some lists of sessions that we think app developers will be interested in.

These lists should help you work out a great schedule!

Look out for the talks Enterprise App Development with Node.js, Python, and PHP for Oracle Database 12c (Monday), Oracle Net Services 12c: Best Practices for Database Performance and Scalability (Tuesday), Meet the Oracle Programming and Scripting Experts (Tuesday), Best Practices for Application Performance and Scalability (Wednesday), and Best Practices for Application High Availability on Thursday.

If you want to explore what else is happening, look at the OOW Content Catalog

Article on installing node-oracledb on Windows

Wed, 2015-10-21 23:14
OTN has posted a detailed article on Installing node-oracledb on Microsoft Windows by Bill Christo (@bchr02). Check it out!

You may also be interested in the research Bill is doing into pre-creating binaries so you don't need to compile yourself: github.com/oracle/node-oracledb/issues/18.