We use Informatica here, but I'm not yet familiar enough with it to decide if I like it or not, or if a well designed cron (update: now using AutoSys to schedule jobs...good reasons to use it, but it would be fun to build a similar system in perl) and perl based system would be just as well. It seems to me that its great advantage/feature (questionable either way) is that it allows non-programmers to set up data transformations in a GUI (i.e. point 'n click) client (if you like that sort of thing). Informatica can handle certain kinds of simple transformations itself (e.g. concatenating strings), but for more complicated things (the example I'm looking at in the book now is converting the hex chars in a URL string to ascii) you can write a 'Perl advanced external procedure transformation' (Update: Can't find reference to that anymore, but I have run into custom transformation written in 'C'...yuck!).

I'm going to guess that there will be times when someone will say "Oh, I can't do that," and you'll be able to say "We can just write an advanced procedure transformation" (don't even mention the perl or external part - until you have it working :-). That's not even counting the pre or post session 'commands' which are mostly unix shell scripts at our site here, but could easily be perl scripts (to, say, email notification of session success or failure, or ftp the results somewhere).

I say just go with it, some manager has decided to blow their budget on this, so once you get a handle on it, it'll be another buzz-word to add to your resume :-)

Update (2008): After several years of using Informatica, I have to agree with mpeppler's friend below. You gain nothing from eliminating "all that Perl and SQL" except a pretty GUI, which can sometimes take a lot of pointing and clicking to make a simple change whereas with perl you just open the file in your favorite text editor and make your simple change. I once found it was easier in Informatica to export objects as XML, edit the XML in Vim, and then import the XML back into Informatica (not for every sort of edit, and not for the faint-hearted).

(update cont.): You still have SQL (and other crap), except now it's scattered across GUI widgets that you have to click and click and click to get to (if you can even find what you're looking for...there's no "grep"). And if people think Informatica cleans up their perl and SQL "mess", well, if you've ever look at the database that Informatica stores its metadata in, then you'd know that you're just trading one sort of mess for another...and with text files at least the information is easier to get to. If someone wants to migrate away from Informatica, let me know, because I'd love to develop a perl ETL framework...though working with Informatica has given me more XML parsing experience than I've ever had anywhere else (though most "developers" that know Informatica will never see the XML)

Update: My favorite ETL rant here.

YA Update: I'll say a couple of good things about Informatica...it does provide built in logging that the developer doesn't have to deal with developing, and nice real-time monitoring of running processes in a prettily packaged GUI. The bad thing is that you'll probably be replaced by someone who can't program their way out of a paper bag.

In reply to Re: Perl and Informatica, or, Applications, like curves, may be open, closed, both, or neither by runrig
in thread Perl and Informatica, or, Applications, like curves, may be open, closed, both, or neither by adamsj

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.