in reply to Re^2: Reading files, skipping very long lines...
in thread Reading files, skipping very long lines...

If you're reading each line as it's appended to the file, you can seek to the end of the file as soon as you see that the line is too long.
Caution: Contents may have been coded under pressure.
  • Comment on Re^3: Reading files, skipping very long lines...

Replies are listed 'Best First'.
Re^4: Reading files, skipping very long lines...
by rir (Vicar) on Sep 30, 2005 at 15:08 UTC
    How do you know if you've skipped over a newline?

    Be well,
    rir

[reply]
      If you're reading each line as soon as it is appended to the file, there won't be a newline.

      But if there's a chance that another line will follow more quickly than your seek gets there, you could always read backwards $MAX bytes to see if there's a newline.

      Caution: Contents may have been coded under pressure.
[reply]
[d/l]
        If you're reading each line as soon as it is appended to the file, there won't be a newline.

        But we are not reading the lines, we are just seeking to the EOF. How do we guarantee that we are fast enough? (I am not sure if the system or just the program is overwhelmed by size of lines.)

        Let MAX be seven. N is a newline. The following data arrives.

        1...N.........9........N.2N...............f....x15

        We capture the line containing 1. We start on the next line but give up before we get to character 9; so we seek on the file and end up at x15; then we read back to about f. How can we ever be sure that we are not missing one or more lines in the area of 2?

        I feel confident there is a race condition but it has been more than a decade since I've seeked on any file in any language.

        Be well,
        rir

[reply]

In Section Seekers of Perl Wisdom