Christopher Jones

Offloading slow batch tasks to an external process is a common method of improving website responsiveness. One great way to initiate such background tasks in PHP is to use Oracle Streams Advanced Queuing in a producer-consumer message passing fashion. Oracle AQ is highly configurable. Messages can queued by multiple producers. Different consumers can filter messages. From PHP, the PL/SQL interface to AQ is used. There are also Java, C and HTTPS interfaces, allowing wide architectural freedom.

The following example simulates an application user registration system where the PHP application queues each new user's street address. An external system monitoring the queue can then fetch and process that address. In real life the external system might initiate a snail-mail welcome letter, or do further, slower automated validation on the address.

The following SQL*Plus script qcreate.sql creates a new Oracle user demoqueue with permission to create and use queues. A payload type for the address is created and a queue is set up for this payload.

-- qcreate.sql

connect / as sysdba
drop user demoqueue cascade;

create user demoqueue identified by welcome;
grant connect, resource to demoqueue;
grant aq_administrator_role, aq_user_role to demoqueue;
grant execute on dbms_aq to demoqueue;
grant create type to demoqueue;

connect demoqueue/welcome@localhost/orcl

-- The data we want to queue
create or replace type user_address_type as object (
  name        varchar2(10),
  address     varchar2(50)
);
/

-- Create and start the queue
begin
 dbms_aqadm.create_queue_table(
   queue_table        =>  'demoqueue.addr_queue_tab',
   queue_payload_type =>  'demoqueue.user_address_type');
end;
/

begin
 dbms_aqadm.create_queue(
   queue_name         =>  'demoqueue.addr_queue',
   queue_table        =>  'demoqueue.addr_queue_tab');
end;
/

begin
 dbms_aqadm.start_queue(
   queue_name         => 'demoqueue.addr_queue',
   enqueue            => true);
end;
/

The script qhelper.sql creates two useful helper functions to enqueue and dequeue messages:

-- qhelper.sql
-- Helpful address enqueue/dequeue procedures

connect demoqueue/welcome@localhost/orcl

-- Put an address in the queue
create or replace procedure my_enq(name_p in varchar2, address_p in varchar2) as
  user_address       user_address_type;
  enqueue_options    dbms_aq.enqueue_options_t;
  message_properties dbms_aq.message_properties_t;
  enq_id             raw(16);
begin
  user_address := user_address_type(name_p, address_p);
  dbms_aq.enqueue(queue_name         => 'demoqueue.addr_queue',
                  enqueue_options    => enqueue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  commit;
end;
/
show errors

-- Get an address from the queue
create or replace procedure my_deq(name_p out varchar2, address_p out varchar2) as
  dequeue_options    dbms_aq.dequeue_options_t;
  message_properties dbms_aq.message_properties_t;
  user_address       user_address_type;
  enq_id             raw(16);
begin
  dbms_aq.dequeue(queue_name         => 'demoqueue.addr_queue',
                  dequeue_options    => dequeue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  name_p    := user_address.name;
  address_p := user_address.address;
  commit;
end;
/
show errors

The script newuser.php is the part of the PHP application that handles site registration for a new user. It queues a message containing their address and continues executing:

<?php
// newuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// The new user details
$username = 'Fred';
$address  = '500 Oracle Parkway';

// Enqueue the address for later offline handling
$s = oci_parse($c, "begin my_enq(:username, :address); end;");
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address",  $address,  50);
$r = oci_execute($s);

// Continue executing
echo "Welcome $username\n";

?>

It executes an anonymous PL/SQL block to create and enqueue the address message. The immediate script output is simply the echoed welcome message:

Welcome Fred

Once this PHP script is executed, any application can dequeue the new message at its leisure. For example, the following SQL*Plus commands call the helper my_deq() dequeue function and displays the user details:

-- getuser.sql

connect demoqueue/welcome@localhost/orcl

set serveroutput on
declare
  name varchar2(10);
  address varchar2(50);
begin
  my_deq(name, address);
  dbms_output.put_line('Name     : ' || name);
  dbms_output.put_line('Address  : ' || address);
end;
/

The output is:

Name     : Fred
Address  : 500 Oracle Parkway

If you instead want to check the queue from PHP, use getuser.php:

<?php
// getuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// dequeue the message
$sql = "begin my_deq(:username, :address); end;";
$s = oci_parse($c, $sql);
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address", $address, 50);
$r = oci_execute($s);

echo "Name     : $username\n";
echo "Address  : $address\n";

?>

If the dequeue operation is called without anything in the queue, it will block waiting for a message until the queue wait time expires.

The PL/SQL API has much more functionality than shown in this overview. For example you can enqueue an array of messages, or listen to more than one queue. Queuing is highly configurable and scalable, providing a great way to distribute workload for a web application. Oracle Advanced Queuing is available in all editions of the database. More information about AQ is in the Oracle Streams Advanced Queuing User's Guide.

Bootnote: The basis for this blog post comes from the Underground PHP and Oracle Manual

This post shows the changes to the ZF2 tutorial application to allow it to run with Oracle Database 11gR2.

Oracle Database SQL identifiers are case insensitive by default so "select Abc from Xyz" is the same as "select abc from xyz". However the identifier metadata returned to programs like PHP is standardized to uppercase by default. After executing either query PHP knows that column "ABC" was selected from table "XYZ".

In PHP code, array indices and object attributes need to match the schema identifier case that is returned by the database. This is either done by using uppercase indices and attributes in the PHP code, or by forcing the SQL schema to case-sensitively use lower-case names.

The former approach is more common, and is shown here.

The instructions for creating the sample ZF2 application are here. Follow those steps as written, making the substitutions shown below.

In Oracle 11gR2, the schema can be created like:

DROP USER ZF2 CASCADE;

CREATE USER ZF2 IDENTIFIED BY WELCOME
    DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS
    TEMPORARY TABLESPACE TEMP;

GRANT CREATE SESSION
    , CREATE TABLE
    , CREATE PROCEDURE
    , CREATE SEQUENCE
    , CREATE TRIGGER
    , CREATE VIEW
    , CREATE SYNONYM
    , ALTER SESSION
TO ZF2;

CONNECT ZF2/WELCOME

CREATE TABLE ALBUM (
  ID NUMBER NOT NULL,
  ARTIST VARCHAR2(100) NOT NULL,
  TITLE VARCHAR2(100) NOT NULL,
  PRIMARY KEY (ID)
);

CREATE SEQUENCE ALBUMSEQ;

CREATE TRIGGER ALBUMTRIGGER BEFORE INSERT ON ALBUM FOR EACH ROW
BEGIN
  :NEW.ID := ALBUMSEQ.NEXTVAL;
END;
/

INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('The  Military  Wives', 'In  My  Dreams');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Adele', '21');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Bruce  Springsteen', 'Wrecking Ball (Deluxe)');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Lana  Del  Rey', 'Born  To  Die');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Gotye', 'Making  Mirrors');

COMMIT;

The driver and credentials are Oracle-specific. Always use the OCI8 adapter in ZF, since it is more stable and has better scalability. Specifying a character set will make connection faster.

 return array(
     'db' => array(
-        'driver'         => 'Pdo',
-        'dsn'            => 'mysql:dbname=zf2tutorial;host=localhost',
-        'driver_options' => array(
-            PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
-        ),
+        'driver'         => 'OCI8',
+        'connection_string' => 'localhost/orcl',
+        'character_set' => 'AL32UTF8',
     ),
     'service_manager' => array(
         'factories' => array(

 return array(
     'db' => array(
-        'username' => 'YOUR USERNAME HERE',
-        'password' => 'YOUR USERNAME HERE',
+        'username' => 'ZF2',
+        'password' => 'WELCOME',
     ),
     // Whether or not to enable a configuration cache.
     // If enabled, the merged configuration will be cached and used in

The rest of the application changes are just to handle the case of the Oracle identifiers correctly.

                     $dbAdapter = $sm->get('Zend\Db\Adapter\Adapter');
                     $resultSetPrototype = new ResultSet();
                     $resultSetPrototype->setArrayObjectPrototype(new Album());
-                    return new TableGateway('album', $dbAdapter, null, $resultSetPrototype);
+                    return new TableGateway('ALBUM', $dbAdapter, null, $resultSetPrototype);
                 },
             ),
         );

 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();

 <h1><?php echo $this->escapeHtml($title); ?></h1>
 
 <p>Are you sure that you want to delete
-'<?php echo $this->escapeHtml($album->title); ?>' by
-'<?php echo $this->escapeHtml($album->artist); ?>'?
+'<?php echo $this->escapeHtml($album->TITLE); ?>' by
+'<?php echo $this->escapeHtml($album->ARTIST); ?>'?
 </p>
 <?php
 $url = $this->url('album', array(
            'action' => 'delete',
-           'id'     => $this->id,
+           'id'     => $this->ID,
        ));
 ?>
 <form action="<?php echo $url; ?>" method="post">
     <div>
-    <input type="hidden" name="id" value="<?php echo (int) $album->id; ?>" />
+    <input type="hidden" name="id" value="<?php echo (int) $album->ID; ?>" />
     <input type="submit" name="del" value="Yes" />
     <input type="submit" name="del" value="No" />
     </div>

         'album',
         array(
             'action' => 'edit',
-            'id'     => $this->id,
+            'id'     => $this->ID,
         )
     ));
 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();

</tr>
 <?php foreach ($albums as $album) : ?>
 <tr>
-<td><?php echo $this->escapeHtml($album->title);?></td>
-<td><?php echo $this->escapeHtml($album->artist);?></td>
+<td><?php echo $this->escapeHtml($album->TITLE);?></td>
+<td><?php echo $this->escapeHtml($album->ARTIST);?></td>
 <td>
 <a href="<?php echo $this->url('album',
-            array('action'=>'edit', 'id' => $album->id));?>">Edit</a>
+            array('action'=>'edit', 'id' => $album->ID));?>">Edit</a>
     <a href="<?php echo $this->url('album',
-            array('action'=>'delete', 'id' => $album->id));?>">Delete</a>
+            array('action'=>'delete', 'id' => $album->ID));?>">Delete</a>
     </td>
     </tr>
 <?php endforeach; ?>

 class Album
 {
-    public $id;
-    public $artist;
-    public $title;
+    public $ID;
+    public $ARTIST;
+    public $TITLE;
     protected $inputFilter;
 
     public function exchangeArray($data)
     {
-        $this->id     = (!empty($data['id'])) ? $data['id'] : null;
-        $this->artist = (!empty($data['artist'])) ? $data['artist'] : null;
-        $this->title  = (!empty($data['title'])) ? $data['title'] : null;
+        $this->ID     = (!empty($data['ID'])) ? $data['ID'] : null;
+        $this->ARTIST = (!empty($data['ARTIST'])) ? $data['ARTIST'] : null;
+        $this->TITLE  = (!empty($data['TITLE'])) ? $data['TITLE'] : null;
     }
 
     public function getArrayCopy()

and

             $factory     = new InputFactory();
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'id',
+                        'name'     => 'ID',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'Int'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'artist',
+                        'name'     => 'ARTIST',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'title',
+                        'name'     => 'TITLE',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),

     public function getAlbum($id)
     {
         $id  = (int) $id;
-        $rowset = $this->tableGateway->select(array('id' => $id));
+        $rowset = $this->tableGateway->select(array('ID' => $id));
         $row = $rowset->current();
         if (!$row) {
             throw new \Exception("Could not find row $id");

and

     public function saveAlbum(Album $album)
     {
         $data = array(
-            'artist' => $album->artist,
-            'title'  => $album->title,
+            'ARTIST' => $album->ARTIST,
+            'TITLE'  => $album->TITLE,
         );
 
-        $id = (int)$album->id;
+        $id = (int)$album->ID;
         if ($id == 0) {
             $this->tableGateway->insert($data);
         } else {
             if ($this->getAlbum($id)) {
-                $this->tableGateway->update($data, array('id' => $id));
+                $this->tableGateway->update($data, array('ID' => $id));
             } else {
                 throw new \Exception('Form id does not exist');
             }

and

     public function deleteAlbum($id)
     {
-        $this->tableGateway->delete(array('id' => $id));
+        $this->tableGateway->delete(array('ID' => $id));
     }
 }

         parent::__construct('album');
         $this->setAttribute('method', 'post');
         $this->add(array(
-                'name' => 'id',
+                'name' => 'ID',
                 'type' => 'Hidden',
             ));
         $this->add(array(
-                'name' => 'title',
+                'name' => 'TITLE',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Title',
                 ),
             ));
         $this->add(array(
-                'name' => 'artist',
+                'name' => 'ARTIST',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Artist',

         }
 
         return array(
-            'id' => $id,
+            'ID' => $id,
             'form' => $form,
         );
     }

and

         }
 
         return array(
-            'id'    => $id,
+            'ID'    => $id,
             'album' => $this->getAlbumTable()->getAlbum($id)
         );
     }

When you create applications from scratch it will be straightforward to get it all working.

In case you missed the notifications, the Oracle Technology Network forum software is going to be upgraded this weekend. This is great, since the old software is getting long-in-the-tooth and doesn't allow a bunch of useful features. The current forums will be in read-only mode over the weekend.

On launch of the new version, a minimal set of features will be supported. Once the upgrade is stable, then additional features will be turned on.

As a side part to the migration project, some little used and obsolete forums will be removed. Some other categories will be reworked.

You can get more information about the migration here. One thing to note is that the forum software is powered by Jive; i.e it is a packaged application so not all features you (and I) have requested will magically become feasible. And also, for better or worse, Jive has renamed "forums" as "spaces" - apparently we can't change this.

The topic of Oracle 11g DRCP connection pooling in Python cx_Oracle came up twice this week for me. DRCP is a database tier connection pooling solution which is great for applications run in multiple processes. There is a whitepaper on DRCP that covers a lot of background and talks about configuration. This whitepaper is ostensibly about PHP but is good reading for all DRCP users.

The first DRCP and cx_Oracle scenario I dealt with was a question about mod_python.

To cut a long story short, I created a handler and requested it 1000 times via Apache's 'ab' tool. In my first script, and despite having increased the default pool parameters, there were a high number of NUM_WAITS. Also NUM_AUTHENTICATIONS was high. Performance wasn't the best. Querying V$CPOOL_CC_STATS showed:

CCLASS_NAME  NUM_REQUESTS   NUM_HITS NUM_MISSES  NUM_WAITS NUM_AUTHENTICATIONS
------------ ------------ ---------- ---------- ---------- -------------------
HR.CJDEMO1           1000        992          8        478                1000

At least the session information in each DRCP server was reused (shown via a high NUM_HITS).

Results were better after fixing the script to look like:

from mod_python import apache
import cx_Oracle
import datetime

# Example: Oracle 11g DRCP with cx_Oracle and mod_python

# These pool params are suitable for Apache Pre-fork MPM
mypool = cx_Oracle.SessionPool(user='hr', password='welcome',
         dsn='localhost/orcl:pooled', min=1, max=2, increment=1)

def handler(req):
    global mypool

    req.content_type = 'text/html'
    n = datetime.datetime.now()
    req.write (str(n) + "<br>");

    db = cx_Oracle.connect(user='hr', password='welcome',
            dsn='localhost/orcl:pooled', pool=mypool, cclass="CJDEMO1",
            purity=cx_Oracle.ATTR_PURITY_SELF)

    cur = db.cursor()
    cur.execute('select * from locations')
    resultset = cur.fetchall()
    for result in resultset:
        for item in result:
            req.write (str(item) + " ")
        req.write ("<br>")
    cur.close()
    mypool.release(db)

    return apache.OK

The 'ab' benchmark on this script ran much faster and the stats from V$CPOOL_CC_STATS looked much better. The number of authentications was right down about to about 1 per Apache (ie. mod_python) process:

CCLASS_NAME  NUM_REQUESTS   NUM_HITS NUM_MISSES  NUM_WAITS NUM_AUTHENTICATIONS
------------ ------------ ---------- ---------- ---------- -------------------
HR.CJDEMO1           1000        977         23         13                  26

The NUM_HITS was high again, because the DRCP purity was ATTR_PURITY_SELF. If I hadn't wanted session information to be reused each time the handler was executed, I could have set the purity to ATTR_PURITY_NEW. If I'd done this then NUM_HITS would have been low and NUM_MISSES would have been high.

If you're testing this yourself, before restarting the DRCP pool don't forget to shutdown Apache to close all DB connections. Otherwise restarting the pool will block. Also, if you're interpreting your own V$CPOOL_CC_STATS stats don't forget to account for the DRCP "dedicated optimization" that retains an association between clients (mod_python processes) and the DB. The whitepaper previously mentioned discusses this.

The second place where DRCP and python came up this week was on the cx_Oracle mail list. David Stanek posed a question. He was seeing application processes blocking while waiting for a DRCP pooled server to execute a query. My variant of David's script is:

import os
import time
import cx_Oracle

# Example: Sub-optimal connection pooling with Oracle 11g DRCP and cx_Oracle

def do_connection():
    print 'Starting do_connection ' + str(os.getpid())
    con = cx_Oracle.connect(user=user, password=pw, dsn=dsn, cclass="CJDEMO2",
           purity=cx_Oracle.ATTR_PURITY_SELF)
    cur = con.cursor()
    print 'Querying ' + str(os.getpid())
    cur.execute("select to_char(systimestamp) from dual")
    print cur.fetchall()
    cur.close()
    con.close()
    print 'Sleeping ' + str(os.getpid())
    time.sleep(30)
    print 'Finishing do_connection ' + str(os.getpid())
 
user = 'hr'
pw = 'welcome'
dsn = 'localhost/orcl:pooled'
for x in range(100):
    pid = os.fork()
    if not pid:
        do_connection()
        os._exit(0)

This script forks off a bunch of processes - more than the number of pooled DRCP servers (see MAXSIZE in the DBA_CPOOL_INFO view). The first few processes grab a DRCP server from the pool and do their query. But they don't release the DRCP server back to the DRCP pool until after the sleep() when the process ends. The other forked processes are blocked waiting for those DRCP servers to become available. This isn't optimal pool sharing.

My suggestion was to use an explicit cx_Oracle session pool like this:

import os
import time
import cx_Oracle

# Example: Connection pooling with Oracle 11g DRCP and cx_Oracle
 
def do_connection():
    print 'Starting do_connection ' + str(os.getpid())
    mypool = cx_Oracle.SessionPool(user=user,password=pw,dsn=dsn,min=1,max=2,increment=1)
    con = cx_Oracle.connect(user=user, password=pw,
          dsn=dsn, pool = mypool, cclass="CJDEMO3", purity=cx_Oracle.ATTR_PURITY_SELF)
    cur = con.cursor()
    print 'Querying ' + str(os.getpid())
    cur.execute("select to_char(systimestamp) from dual")
    print cur.fetchall()
    cur.close()
    mypool.release(con)
    print 'Sleeping ' + str(os.getpid())
    time.sleep(30)
    print 'Finishing do_connection ' + str(os.getpid())

user = 'hr'
pw = 'welcome'
dsn = 'localhost/orcl:pooled'
for x in range(100):
    pid = os.fork()
    if not pid:
        do_connection()
        os._exit(0)

The mypool.release(con) call releases the DRCP server back to the DRCP pool prior to the sleep. When this second script is run, there is a smoothness to the output. The queries happen sequentially without noticeably being blocked.

Like with any shared resource, it is recommended to release DRCP pooled servers back to the pool when they are no longer needed by the application.

Zend have contributed their Zend Optimizer+ opcode cache to PHP - thanks Zend!!! (Update 19 March 2013: the renaming to "Zend OPcache" is complete)

"The Zend OPcache provides faster PHP execution through opcode caching and optimization."

The new OPcache extension can be seen as substitute for the venerable APC cache, the maintenance of which had become an issue. Note: although OPcache is now readily available, there is currently nothing preventing you from using any available (working!) opcode cache in PHP 5.5.

A few minutes ago Dmitry Stogov did the physical merge to the PHP 5.5 source's ext/opcache directory. The current PHP 5.5 snapshot has the code. Future Alpha or Beta (and Production) releases will include it too.

Please test OPcache. It is not a panacea for all performance problems. There are a lot of settings which may need adjusting. Understanding how it works and identifying issues during the stabilization phase of PHP 5.5 release process will greatly help.

To build Zend OPcache for PHP 5.5:

When you configure PHP, add the option --enable-opcache like:

./configure ... --enable-opcache

Then make and make install, as normal. This will build OPcache shared extension. It's not possible to build it statically.

Find the shared library in your installation directory with a command like find /home/cjones/php55 -name opcache.so

Edit php.ini and add the extension with its full path:

zend_extension=/home/cjones/php55/lib/php/extension/debug-non-zts-20121212/opcache.so

Update (25 March 2013): Dmitry merged a PHP 5.5 change so that the full path is not required for zend_extension libraries in the extension_dir directory. You can now simply do zend_extension=opcache.so.

You'll want to enable OPcache too:

opcache.enable=On

The ext/opcache/README is the current source of documentation, and lists all the other php.ini parameters.

Problems can be reported in the Github issue tracker

Update (18 March 2013): In a commit over the weekend, the build option --enable-opcache is On by default. You will still need to update php.ini.

To build Zend OPcache for older versions of PHP:

You should be able to build OPcache with PHP 5.2 onwards

Install it by getting the source from Github. There is also a PECL repository; this is slightly out of date so I recommend using Github. Follow the README instructions to install it.

User-data cache:

The new opcode cache does not include a user-data cache. Joe Watkins recently started the APCu project to extract the user-data cache code from APC. Test this too!

Update 19 March 2013: Xinchen Hui is working on a lockless user-data cache, see https://github.com/laruence/yac and his blog about it here (in Chinese).

The PHP RFC process has been in place for a while, and users new to core PHP development are starting to use RFCs to propose desirable features.

Here are some personal observations and suggestions that show how I have seen feature acceptance and the RFC process work in practice. These notes augment the steps in How To Create an RFC. I hope they help set expectations about the PHP RFC process and feature acceptance in the PHP language.

• Before starting an RFC, review existing RFCs and search the mail list archives for similar ideas. Use this information to explain what your RFC will bring to the language.

• If you're new to PHP core development, mail the "internals" mail list to judge interest before you start your RFC. If you get no reaction (this is likely) or positive reaction then create the RFC. Core PHP developers with karma will know when they can skip this step.

• Avoid presenting an RFC idea to the "internals" mail list with email that begins "I don't know much about ... but ...". Do some research first.

• There are many really good ideas for improving PHP, however some of them are really tedious or technically risky or hard. If you are about to email the "internals" mail list saying "someone should do ...", then don't hit "Send". Work out how you could do it, and then send a patch.

• If the overall goals of PHP are not clear to you, you may not be alone. However, as an RFC writer you need to learn the general trajectory of the language and where web development is heading. You then need to enthuse everyone with your feature and show what it brings to the language.

• Don't start an RFC (or mail list discussion) about standardizing PHP function names and function argument orders. There are several historical reasons why the functions are what they are (including compatibility with underlying libraries), and many good reasons why the change would be counter-productive causing more end-user confusion, would lead to unmaintainable PHP engine code, and generally be a waste of everyone's time when they could be doing more interesting projects. This has been discussed ad infinitum. Review previous discussions and feel free to fork PHP on github.

• Your RFC should talk about all PHP areas that will be affected: php.ini, different SAPIs, engine, extensions, etc. List similar features. List similar features in other languages. Link to references. Give an estimate of the actual positive impact to user code. Put the proposed voting options in the RFC so they can be included in its discussion.

• Your RFC will be used to create the PHP documentation, so make its sections re-usable. Explain concepts and details. Keep the RFC technical and have plenty of examples.

• Many current PHP RFCs don't contain enough detail, nor do they capture the pros & cons that were discussed on the mail list. You can, and should, do better than these RFCs.

• An implementation is not mandatory while proposing an RFC but it can help persuade people and help fine-tune the design. Note that if you are unable to implement the RFC, and no one else volunteers during the discussion period, then your RFC is unlikely ever to be implemented.

• Don't start an implementation too early. Get support for the feature concept from the "internals" mail list before spending time on coding - unless you want a learning experience.

• If you do have an implementation, make it clear whether the implementation is a simple prototype or is expected to be the final code. This is specially important during the vote.

• It's easy to get sidetracked by mail list trolling or well intentioned but over-eager discussion. Take advice on board but make sure your feature doesn't end up being designed by a committee. Don't blame the "PHP core community" for derailing discussions when the issues of non- code-contributors on the mail list are at fault.

• For every user who wants a backwardly incompatible change to PHP, there is a user who will scream that this will be disastrous to them. As the RFC writer you need to walk the line between these groups. At voting time, other people may see the line in a different place than you do.

• There is no need to respond to every discussion email individually. You should batch up your responses and manage the discussion intelligently.

• Don't get side tracked by how long it will take a feature to reach end users. It will reach them eventually. You can always supply an unofficial patch or maybe create a PECL extension for users of current versions of PHP.

• Don't let mail list discussion drag on too long. High levels of discussion can be a symptom that an RFC is contentious. Having an early vote about only the feature's concept can prevent over-investment in an RFC and implementation.

• With long, fragmented discussions, not everyone will read every email. Update the RFC at regular intervals, and let people know what has changed.

• PHP is popular so you can often find support from "many" people for cool language features. This doesn't mean those features are a "good thing" for the language or its implementation.

• There are multiple PHP communities. Don't interpret support in one community as universal support. Listen carefully to the key PHP developers on the "internals" mail list (they may not be the ones doing the most discussion). They are the people who will undoubtedly end up doing final integration of the feature (e.g. with the op-code cache), fixing edge cases, and maintaining it in the long term. They are the ones that have seen the PHP code base complexity grow and become more difficult to maintain and extend. Even if they like your idea, they may not be able to contribute time and effort to make it stable.

• The PHP core development community needs more end-users to become core language developers but the barrier to entry is high because of the complexity of the API, and because new features need very careful integration with a complex language. See the point above about existing developers not having enough time to support every good idea. This just means you need perseverance to become a core developer yourself. You can gain karma by being a regular code contributor before starting your magnum opus RFC.

• Some areas of PHP are complex or niche. Sometimes feature suggestions will be greeted by an apparent lack of interest. Don't be discouraged. This just means you need to take a stronger leadership role, and also prove your credentials by first working on the existing code base.

• Keep your RFC updated with a summary of all the positive and negative mail list discussion points and examples.

  • This prevents arguments being rehashed on the mail list each year.
  • Documentation will have clear use cases and show what does and doesn't work.
  • Future RFC writers can see trends and build on previous ideas.

• Before initiating a vote on your RFC, make sure the RFC contains enough detail so that the vote is solely on the RFC contents, not on any half-remembered or misinterpretable mail list discussions. Un- (or badly) expressed intentions will only cause you frustration when your RFC is rejected.

• Don't leave any ambiguity in the RFC. As well as stating what will be changed, also state what won't be changed. Ambiguity will hurt the chances of a successful outcome because voters will be unsure that the feature has been fully thought through. Make sure there are no "Open Issues" in the RFC when voting. Make a decision about the choices before opening the vote, or mark the issues clearly as "Future topics for exploration".

• Make sure any vote is clear about what is being voted on: is it the idea, the specification, or the implementation? Is a particular version of PHP being targeted? Is the implementation going to be merged to the core language or made as a PECL extension?

• Warn the "internals" mail list before starting the vote. This notification gives you a chance to fine tune the wording of your poll; this wording has caused contention in the past. The approaching deadline may also cause last-minute RFC responses. These may be significant enough to require further discussion.

• Set a deadline for the vote (sadly the current voting RFC doesn't require this). Having a deadline will forestall suggestions that a vote was deliberately left open until it succeeded, and then closed before the "negative" side could rally support.

• During the voting period, it is common for people to continue mail list discussion. You may need to halt the vote and address any issues.

• Prior to the end of the voting period, remind the mail list that the vote is about to close.

• Positive voting poll outcomes for RFs that are just "concepts" can be interpreted as support for "good ideas" rather than being a definitive guarantee that a feature will appear in a future version of PHP. As a feature is later investigated and further discussed, the early vote may become irrelevant. Similarly, where there is no chance of an implementation being written, a positive poll outcome is just an indicator of desire.

• If your RFC is rejected, it is not the end of the world. Sometimes ideas are submitted "before their time", and it takes an acclimatization period for other developers to see their value. As the core PHP developer base goes through natural turn-over, revisiting an idea after a (long) period may result in different appreciation. (This does not apply to some ideas, for example standardizing function names). See the previous points about becoming a core contributor - having karma goes a long way towards getting an RFC accepted, not only because experienced contributors know which RFCs are worth suggesting, and know how to make proposals understandable. When your RFC is rejected, continue working with the PHP core development community and revisit the RFC later.

• You can always fork PHP from github.

In summary, the PHP development process is an organic process and subject to flexibility. It is based heavily around users contributing features that they require. This therefore requires high investment from users who want change. Very rarely do PHP core developers have bandwidth to take on external ideas, no matter how good they are. Feature acceptance has to be pragmatic. The core PHP contributors would love to see more people get commit karma and contribute to PHP.

This post has gone on long enough. My notes are current for the time being. I'm sure there are observations or clarifications you can make. If you want to add anything, please post a comment.

(Note: this page has been augmented and re-ordered since first being published)

The Oracle 11g Release 2 (11.2.0.3) Database Instant Client for Apple OS X on Intel x86-64 is now available for download from OTN. It is supported on the two latest OS X releases: Lion (10.7) and Mountain Lion (10.8). It provides both 32-bit and 64-bit client support.

Oracle Instant Client is a simple bundle of libraries that client tools and programs (like PHP and Ruby) can link with. This allows those tools to access Oracle Databases.

Any issues with Instant Client can be posted to the OTN Instant Client Forum.

Who says direct marketing doesn't work? A personal request to blog the upcoming ConFoo conference (25 February - 1 March 2013 in Montreal, Canada) has, as you can see, been successful. Although it's been a few years since I spoke there, I recall how impressive the organization was. The diversity and continual growth trajectory of the conference over the years is a very good indicator that you should be involved.

While you're there, say Hi to Oracle Ace Director Sheeri Cabral who is giving a couple of MySQL talks.