I have no idea how seek could be able to set $. without lots of magic.

It doesn't update $. itself; it does update PL_last_in_gv.

open my $fa, '<', '/etc/passwd' or die $!;
open my $fb, '<', '/etc/group' or die $!;
<$fa>; $.=9; <$fa>; warn "One";
<$fb>; $.=17; <$fb>; warn "Two";
seek $fa,256,1; warn "Three";
seek $fb,256,1; warn "Four";
<$fa>; warn "Five";
<$fb>; warn "Six";
__END__
One at - line 3, <$fa> line 10.
Two at - line 4, <$fb> line 18.
Three at - line 5, <$fa> line 10.
Four at - line 6, <$fb> line 18.
Five at - line 7, <$fa> line 11.
Six at - line 8, <$fb> line 19.
[download]

See pp_sysseek in pp_sys.c.