If, as you say, you never mix buffered and unbuffered I/O, and never alter $/, then there's something fishy and unexpected. I sincerely doubt that seek() calls, even if done on a buffered stream and even with mixed dos and unix newlines, would produce such an effect. I'd investigate further to find what causes the slurping, at least to make clear if that's a perl bug or not.
otoh, if you're only interested in a practical solution, just switch to read($f, $buf, 41) instead of readline, which depends on $/.