I was wondering if there is a way, without maybe having to read every line in the file, to quickly see if there exist lines that do not start with # (which would mean that I can process this file further.

Not really (but may the other monks correct me if I'm wrong). Since you can't know in general where a new line starts without having read all of the preceding one, you cannot know whether any line starts with a hash mark unless you read them all.

This is for the general case, of course. If the output of your program is further constrained, inferences may be possible; for instance, if you know that every line is precisely 70 characters (including the trailing newline), you could read every 70th character only. Whether that'd be much faster in practice is another question, one that only profiling could answer.

Is there any way that the program generating this output can signal to you whether there's data that needs further processing? If you can change that one's implementation, perhaps have it quit with an appropriate exit code, or drop a marker file, or something along those lines.

In reply to Re: A fast way to do this? by AppleFritter
in thread A fast way to do this? by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.