gnubbs has asked for the wisdom of the Perl Monks concerning the following question:

Monks, I have a very simple question for you about editing files in place. I have played with this a bit, and to me it looks like it should work, but it doesnt:
# Now, we need to add the user to the fetchmail files... open(FETCHMAILRC, '+< .fetchmailrc') or die "Unable to open FMRC."; while (<FETCHMAILRC>) { $_ =~ s/^here/$username\nhere/i; }
Your insight will be greatly appreciated. gnubbs

Replies are listed 'Best First'.
•Re: Editing Files In Place
by merlyn (Sage) on Jun 13, 2002 at 00:00 UTC
      Oooh purdy. It'd always bugged me that one "had" to use -i. It's bit unfortunate you have to go the whole ARGV/STDIN route, but that's better than forking. thanks!

      --
      perl -pew "s/\b;([mnst])/'$1/g"

        the whole ARGV/STDIN
        Excuse me? There's no use of STDIN in that program. An empty diamond is the same as <ARGV>, not <STDIN>.

        And beginning Perl hackers who confuse the two will eventually get burned. My rule is to never use <> (aka <ARGV>) if you ever print some kind of prompt, and I flunk any use to the contrary in code review. Work it out for yourself to see why. The proper use for interaction where you prompt and read input is <STDIN>.

        -- Randal L. Schwartz, Perl hacker

      I just tried this on perl 5.005_02 and I found that it would not work with local *ARGV but local @ARGV worked.

      I assume localizing the ARGV filehandle somehow broke the <ARGV> magic. It works correctly on perl 5.6.0.

Re: Editing Files In Place
by jarich (Curate) on Jun 13, 2002 at 00:55 UTC
    It looks like it should work...

    In this small snippet of code you never print your changes back to the file. You read a line in, manipulate it and throw it away. Remember that $_ holds the contents of the line. It is not the line itself.

    Your immediate thought might be to add

    print FETCHMAILRC $_;
    to the bottom of your while loop, but this won't work either. It won't work because your file pointer telling you where you've read up to has been incremented to the end of the line you've read. So printing out what you've just read will duplicate it in the file.

    What is more, when you print this line back to the file you move the file pointer on again, so your file will end up completely corrupted. A way to get around that is to use seek lots, but that's beginning to get masochistic.

    If you don't like merlyn's excellent suggestion (although who wouldn't?) you can also do it this way:

    open(FETCHMAILRC, '< .fetchmailrc') or die "Unable to open FMRC for re +ading. $!"; my $filecontents; while (<FETCHMAILRC>) { s/^here/$username\nhere/i; $filecontents .= $_; } close FETCHMAILRC or die "Close on FMRC died: $!"; # clobber file and open for writing open(FETCHMAILRC, "> .fetchmailrc') or die "Unable to open FMRC for wr +iting. $!"; print FETCHMAILRC $filecontents; close FETCHMAILRC or die "Close on FMRC died: $!";
    Note that I snuck $! into those die statements too. $! tells you what just went wrong, so rather than just being told that you were unable to open FRMC for reading you'll be told why: file doesn't exist, permission denied etc. If there is any chance whatsoever that more than one process might try to do this at a time then you really must look into locking. There are many excellent nodes around here discussing locking in detail.

    Hope it helps.

    jarich

Re: Editing Files In Place
by Abigail-II (Bishop) on Jun 13, 2002 at 09:42 UTC
    Editing files inplace is tricky. One has to remember than on most platforms, including UNIX and Windows, files are just sequences of bytes. There is no line based datastructure. Your approach isn't going to work, unless the replacement part is the same length (in bytes) as what you are replacing. But then you are still missing a seek (you need to seek back to reposition the file pointer) and a print.

    There are two often used approaches for generic modifications:

    • Read in the entire file into memory. Make the modifications, seek back to the beginning of the file, and print the modified content. Don't forget to perform a truncate, otherwise you may end up with trailing garbage if the modified text is smaller than the original. Obviously, this approach works better for small files than huge files.
    • Use a temp file. Read in parts (for instance lines) of the original file, do the modifications, and write the modified text to the temp file. If you are all done, move the temp file to the original file.

    You may want to use Tie::File, or investigate the -i option of perl itself (or its $^I equivalent).

    Abigail

Re: Editing Files In Place
by yodabjorn (Monk) on Jun 13, 2002 at 00:44 UTC
    Randal you are the man :-D
    never really thought about the <>  vs. <STDIN> I just assumed STDIN

    Thnx once again for being so informative ..

Re: Editing Files In Place
by jmcnamara (Monsignor) on Jun 13, 2002 at 07:33 UTC

    I may be missing some additional context but editing the file in-place seems to be an overly complicated way of adding a new username. Your code and comments suggest that what is really required is to append to the file.

    In which case you could do something like the following (see also perlopentut):

    open(FETCHMAILRC, '>> .fetchmailrc') or die "Unable to open FMRC." +; print FETCHMAILRC $username, "\n";

    If you really need to edit the file in-place then you can use the -i command line option or $^I, as demonstrated so elegantly by merlyn.

    --
    John.

Re: Editing Files In Place
by gnubbs (Beadle) on Jun 13, 2002 at 22:39 UTC
    Thanks everyone for your help, especially merlyn, jarich, and abigail-II. One more script done, and a little bit better understanding of how file IO works in perl.