Maelstrom has asked for the wisdom of the Perl Monks concerning the following question:

So apparently when you open a file read/write, then read from it you need to seek back to the beginning and preferably truncate it. This is well documented on countless guides but it's never mentioned what happens if you don't seek back to the start. Would writing to a file without the seek just append the write to the file and if so what is the point of a read/write/append mode? And in a similar vein what would be the point of opening a file in read/write/truncate (w+) mode possibly be? I'm writing a collection of functions with plain English names that abstracts away the use of filehandles so I never have to think about file i/o again but can't for the life of me think of a scenario where either a+ or w+ would be useful.
  • Comment on What is the point of a+ and w+ when opening files?

Replies are listed 'Best First'.
Re: What is the point of a+ and w+ when opening files?
by haukex (Archbishop) on May 17, 2023 at 14:20 UTC
    And in a similar vein what would be the point of opening a file in read/write/truncate (w+) mode possibly be?

    Yes, since w+ mode clobbers the file first, this is not usually a particularly useful mode. But a+ and r+ certainly do have their uses, for example, if you want any kind of random access to read/write the data in the file (e.g. updating fixed-length records) - though personally I've never done this directly, as this is usually the job of a library that handles that particular data format. However, see the last code example in my node here, in particular in combination with the notes on flock above it, this can be useful in some cases.

      though personally I've never done this directly, as this is usually the job of a library that handles that particular data format.

      Ah, yes, the luxuries of modern desktop computing. I grew up with Commodore computers (VIC20, C64, Amiga), and nowadays i sometimes work with Microcontrollers of the Arduino family. When you are low on memory (="always"), you tend to code as low level as possible.

      Heck, for one of my projects i'm talking to the memory chips directly via I2C, because the corresponding library would require ~200(!!!!!) bytes of RAM. When you only got 2048 of those buggers in total, you tend to turn into Scrooge McDuck.

      But even in some of my more recent data extraction projects i had to use random access for fixed size records.

      PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
      Also check out my sisters artwork and my weekly webcomics
Re: What is the point of a+ and w+ when opening files?
by hv (Prior) on May 17, 2023 at 15:35 UTC

    A recent case where I have opened a file "a+" is for a (C) program writing details of progress on a long calculation to a logfile. On startup, if the logfile already exists, I parse what has been written to discover the point from which the calculation should be resumed.

    (Since then I have changed it to open "r" for the parsing, then reopen "a" for normal operation - the "a+" logic worked fine for me on Linux, but appeared to cause problems for others on Windows (via CygWin).)

Re: What is the point of a+ and w+ when opening files?
by NERDVANA (Priest) on May 17, 2023 at 20:10 UTC
    Opening a file w+ is for creating a new blank file (or clobber the existing one if any) that you intend to use as read/write storage. a+ is when you want to use the contents of the file that are already there. You don't have to seek back to the beginning to read after a write, just seek anywhere, so it's useful for structured file databases. Of course, you would probably use a library for that, so it's more common to use it in C than in Perl.

    An example use for Perl would be if you wanted to implement Capture::Tiny (capture your own stderr or stdout). You create a temp file with mode w+ and then redirect stderr into that file, run some code that may generate output, then switch stderr back to the original file handle and read from the temp file to see what got written inbetween. On Unix, if you open that file w+ you can unlink it right after creating it so it doesn't appear in the filesystem (and can't get orphaned if the process aborts) and yet still read/write the file just from that one handle. ...but most people doing this would probably use File::Temp instead of a new set of library functions that create w+ files, so maybe still not a reason for you to implement a helper function to wrap it.

    But, on that note, File::Temp is probably using mode w+ for all the files it creates. So there's another reason for w+, if you want to create the most useful generic file for someone else to use and you don't know what mode they actually need.

    ...And that reminds me of another use - you could be receiving data from an external source and not want to risk filling main memory and crashing the program. So a framework like Catalyst reads packets from the client and writes them to a temp file, then provides that file (after seeking back to byte 0) to the application to read and be able to seek on it, or rename it into a new location since it is already on disk.

Re: What is the point of a+ and w+ when opening files?
by kcott (Archbishop) on May 17, 2023 at 14:29 UTC

    G'day Maelstrom,

    It seems like you are comparing Perl with some other language. If you tell us what that is, we can probably provide a better answer. Links to (some) "countless guides" would no doubt also help. The following may be sufficient for your needs; if not, please provide the additional information that I've indicated.

    Perl uses different tokens for a+, w+, and so on; see "open: About modes".

    That documentation agrees with you. Here's an extract (+< corresponds to r+; +> to w+):

    "... +< is almost always preferred for read/write updates--the +> mode would clobber the file first."

    I don't think the requirement you describe for seek and truncate exists in Perl. Having said that, Perl does have seek(), tell() and truncate().

    — Ken

      Hi Ken not sure I understand your reply as I'm not having a specific problem with PERL or disagreeing with a document, just wanting clarification on file handle modes which I imagine is not specific to PERL. I've been a native (if infrequent) speaker of PERL for about 20 years now and have thus lost the ability to compare PERL with one of the saner languages but it's true I began this library out of frustration when I originally went from DOS/Pascal to Unix/Perl. I started polishing it again when I discovered wantarray and the fact I could initiate a hash by passing it an array. On the one hand typing my %hash = file2hash($file) is very satisfying but on the other, testing (defined wantarray) to return a scalar is truly disgusting to anyone that's ever tried to learn PERL by reading it.

      As for the guides a fairly representative one is at https://perlmaven.com/open-to-read-and-write . As you can see seek and truncate are requirements in that use case but it doesn't go on to explain if you can replicate the effect of opening a+ by not doing them (or why you would die with a message about mailboxes but I digress :)

      In any case the replies have somewhat illuminated the case for these weird file modes so I've made them optional parameters to the write_open sub which seems preferable to writing multiple routines.

        Firstly, just to clear up a few things:

        • I thought you were comparing with another language because you wrote w+ instead of +> and a+ instead of +>>.
        • I didn't think you were looking for a problem solution; more of an explanation.
        • I didn't say you were disagreeing with anything; I said that the documentation was agreeing with you.
        • Perl is the language; perl is the program that runs Perl code; PERL is not a thing. :-)

        In https://perlmaven.com/open-to-read-and-write:

        • The file is opened: open my $fh, '+<', $file — position is 0.
        • The first record is read: <$fh> — position is >0.
        • The filehandle is repositioned to the start: seek $fh, 0, 0 — position is 0.
        • The file contents are removed: truncate $fh, 0 — position is 0.
        • New content is added: print $fh $count — position is >0.

        The only reason that seek() was used here was to reposition to the start. Note that truncate() does not change the position. There is no inherent requirement to use seek() or truncate() after opening a file in one of the read/write modes.

        The die() message with "mailbox" is an error; probably a copy-paste typo. Hand-crafting I/O messages is tedious and error-prone. I prefer to leave that task to Perl with the autodie pragma; it's less work for me and it won't produce bogus or incomplete messages; I recommend that you use this too.

        I threw together a short script (read_write_append.pl) to demonstrate the three read/write modes:

        #!/usr/bin/env perl use 5.010; use strict; use warnings; use autodie; { say "\n*** Test +< (r+)"; say 'Open file'; open my $fh, '+<', 'test1.txt'; say 'Position in file: ', tell $fh; say 'Read to end of file'; print while <$fh>; say 'Position in file: ', tell $fh; } { say "\n*** Test +> (w+)"; say 'Open file'; open my $fh, '+>', 'for_del.txt'; say 'Position in file: ', tell $fh; say 'Read to end of file'; print while <$fh>; say 'Position in file: ', tell $fh; } { say "\n*** Test +>> (a+)"; say 'Open file'; open my $fh, '+>>', 'testq.txt'; say 'Position in file: ', tell $fh; say 'Read to end of file'; print while <$fh>; say 'Position in file: ', tell $fh; say 'Write a record'; say $fh 'r'; say 'Position in file: ', tell $fh; say 'Rewind to start of file'; seek $fh, 0, 0; say 'Position in file: ', tell $fh; say 'Read to end of file'; print while <$fh>; say 'Position in file: ', tell $fh; }

        I created some test files:

        ken@titan ~/tmp/pm_11152251_file_rwa $ ls -l *.txt -rw-r--r-- 1 ken None 6 May 19 01:41 for_del.txt -rw-r--r-- 1 ken None 6 May 19 01:16 test1.txt -rw-r--r-- 1 ken None 6 May 19 01:40 testq.txt ken@titan ~/tmp/pm_11152251_file_rwa $ cat test1.txt 1 2 3 ken@titan ~/tmp/pm_11152251_file_rwa $ cat for_del.txt z x c ken@titan ~/tmp/pm_11152251_file_rwa $ cat testq.txt q w e ken@titan ~/tmp/pm_11152251_file_rwa $

        Ran the script:

        ken@titan ~/tmp/pm_11152251_file_rwa $ ./read_write_append.pl *** Test +< (r+) Open file Position in file: 0 Read to end of file 1 2 3 Position in file: 6 *** Test +> (w+) Open file Position in file: 0 Read to end of file Position in file: 0 *** Test +>> (a+) Open file Position in file: 6 Read to end of file Position in file: 6 Write a record Position in file: 8 Rewind to start of file Position in file: 0 Read to end of file q w e r Position in file: 8 ken@titan ~/tmp/pm_11152251_file_rwa $

        Then checked the files again:

        ken@titan ~/tmp/pm_11152251_file_rwa $ ls -l *.txt -rw-r--r-- 1 ken None 0 May 19 01:42 for_del.txt -rw-r--r-- 1 ken None 6 May 19 01:16 test1.txt -rw-r--r-- 1 ken None 8 May 19 01:42 testq.txt ken@titan ~/tmp/pm_11152251_file_rwa $ cat test1.txt 1 2 3 ken@titan ~/tmp/pm_11152251_file_rwa $ cat for_del.txt ken@titan ~/tmp/pm_11152251_file_rwa $ cat testq.txt q w e r ken@titan ~/tmp/pm_11152251_file_rwa $

        I suggest that you play around with that code to test whatever other scenarios might interest you.

        — Ken

Re: What is the point of a+ and w+ when opening files?
by Marshall (Canon) on May 18, 2023 at 00:12 UTC
    If you ever do use a r/w file, be aware that when you change modes (between write and read), an intervening seek is required to cause the buffers to be flushed to disk. You will see "seek 0 bytes from current position" in the code, that is a no-op functionally but causes a buffer flush.

    A weird kind of file that you probably won't see is called a "sparse file". But this weird thing sometimes has its uses. I have made these both on Unix and on Windows. A seek doesn't necessarily leave physical blank space. If you start at the beginning if a file, write a few bytes, seek 1 GB, write a few bytes, seek 1 GB more, write a few bytes, the file on the disk will typically take about 16 KBytes (2 allocation units), not 2 GB. However, if you try to "cat" that file, it will be 2 GB long because the zeroes will be added. One of the big problems with a file like this is how to back it up! That is because a normal copy will "fill in the holes". Anyway, there are applications for a Huge address space that is only sparsely populated.