Per your request, I am going to focus on how to leverage Perl to make this script run quickly. There are a lot of little stylistic things that would make the code both shorter and more legible. I'm not the style police, though, and won't offer stylistic advice unless you would like to have it.

Looking at your implementation, your code only makes sense if you are working on accounts that have already been sorted by account number. If that's not the case, then I can't see how this would work at all. Furthermore, the control file would contain a subset of accounts to transfer (otherwise you wouldn't be looping through rows in LoanAccountFileHandle) and each account is associated with only one loan, or the control file has multiple lines repeating the same account number, one for each loan. This seems like a brittle approach, but who cares, it's a one-time transition script, and presumably you have a robust method for creating the control file.

With those assumptions in place, it looks to me like your algorithm should scale roughly linearly with the number of lines processed. Why would it slow down? The only thing I can come up with is that you're hitting memory issues on your machine. This could arise because either the Excel writer or the CSV reader are leaking memory (not that they're intrinsically leaky, it's just my guess). When your script is running, what does the RAM consumption look like?


In reply to Re: Optimization of script by dcmertens
in thread Optimization of script by JulioRD

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.