Perhaps this is a silly question, but I have to ask it.

Why do you need to speed up this process? Don't you have to get it right once only? After that (I presume) you would be done.

If you are wondering how to do your testing without going through the whole file, I would cut out a 1000 line chunk of the file (say) and use that for my testing. Once most everything looks good, run your script against the whole file.

As far as I know, some very smart people have worked on getting Perl's "Read in a line and process it" code to run absolutely as fast as possible -- so I don't think tinkering with the buffering is going to have a significant impact on the speed.

How long does it take to run the entire file? Can you get access to a faster machine, or one with more memory?

--t. alex
but my friends call me T.

In reply to Re: Formatting a large number of records by talexb
in thread Formatting a large number of records by elbow

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.