For converting large datasets, my usual rule is "Use SQL when you can, use Perl when you have to". Perl is terrific for coding transformations (more fun too), but there's just no substitute for doing work in the server, without moving the data out and back in.

The last couple of times I've had to do things like this, I developed the transforms in SQL, but used Perl to control the execution. It's a good combination that allows you to exploit the strengths of both tools.

Something else to consider: if this is new territory for you, use it as an opportunity to get good with SQL. Try to avoid cursors and row-by-row processing: use set-oriented operations as much as possible. Avoid "procedural SQL".

Another tip: Back up everything before you start! ;-)

HTH

In reply to Re: Data Migration Perl style vs. SQL style.... by VSarkiss
in thread Data Migration Perl style vs. SQL style.... by mr.dunstan

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.