Unless the lines are fixed length or you cache line start indexes, you can't.

If the file is growing then it may be worth keeping a second file that caches line start indexes every few hundred lines. Should be low overhead and easy to implement.