Re: Rants. Difficulty to learn ETL tools?

From: Peter Nolan <peter_at_peternolan.com>
Date: 23 Apr 2007 02:22:20 -0700
Message-ID: <1177320140.017358.118530_at_e65g2000hsc.googlegroups.com>


Hi DBA,
a couple of other comments....

  1. Skills are not particularly transferrable between the leading ETL tools like Informatica and DataStage....and worse, when people learn one or the other they become 'religiously convinced' that the one they learned is the best and tend to be worse than a newbie if they are forced to transfer to the other.....just human nature...
  2. The ETL tools are evolving rapidly and so there is significant need to stay up to date.....just like any other piece of complex software.
  3. ETL tools make the 'simple things simpler' and the 'harder things just about impossible'. A few years back I was working on a project where the ETL tool forced changes to the data model because the tool could not load the data model as designed. If that is not the cart before the horse I do now know what is.
  4. By making the 'simple things simpler' it has been possible to outsouce more development to lower skilled people. And a lot of that has gone to India. There are some things workable and some unworkable about this.....if you are outsourcing your ETL development then you need at least one good ETL person on site as an employee so you know you are getting 'good stuff' back.
  5. The ETL tools are generic general purpose tools that are then in some way 'coded'. They exhibit the exact same properties of other source code environments...it makes no difference that the 'source code' is presented as squares joined by lines in a GUI...it is still source code of a fashion...it still needs to be tested every time something changes....

And just as an amusing aside....I recently did a project where we used our ETL tools to build the prototype and then recoded everything into Informatica. We hired one of the leading Indian IT companies to do the conversion of our prototype ETL to Informatica.....their lead person looked at what we were giving him and the first question out of his mouth was 'Why do you need us? You already have a working ETL subsystem?' to which the answer was 'Well, the corporate standard is Informatica, and your job is to convert this into Informatica. And you get paid for doing so.'....and they were happy campers.... ;-)

Best Regards

Peter

www.peternolan.com Received on Mon Apr 23 2007 - 11:22:20 CEST

Original text of this message