What you are trying to do can't work, not because of Perl, but because of the very nature of sequential files.

I'm afraid you have to bite the bullet and, one way or another (explicitly or implicitly), copy the whole file.

Edit: fixed a typo (s/byte/bite/). And BTW, perl -pi ... and sed -i ... are just copying the file behind the scene, which is why you perceive them as very slow.