Re: Add a line to the end of a file

Append operations don't read the file into memory. They just set the write pointer to the current end of the file.

Some OSes and file systems have problems reading files above 2 or 4 GB (usually 2). This is the limit of 32-bit file pointers. To get bigger files, you need a system which supports 64-bit file pointers, which puts the size limit many magnitudes larger than any currently available storage device can hold. Win2k and Linux >=2.4.0 supports this. I can't help you about other systems. You'll also need a version of perl compiled to support 64-bit file pointers, but chances are that if your system supports them already, so is your current perl version.

----
: () { :|:& };:

Note: All code is untested, unless otherwise stated

Comment on Re: Add a line to the end of a file Download Code

Replies are listed 'Best First'.
Re: Re: Add a line to the end of a file by waswas-fng (Curate) on Feb 17, 2004 at 22:35 UTC
Solaris 2.6 and later (SunOS 5.6 -> 5.9) can handle files larger than this size as well. If you are using veritas filesystem you may need to make sure you are mounting with the largefiles option. The only other problem you will want to avoid here is to make sure that you don't have any race conditions if you run the append on multiple threads or processes at the same time. You will want to look into locking in that case, or maybe a service that handles writing to the file and takes the input from a port or queue (think like syslog or sendmail). -Waswas	[reply]
Re: Re: Re: Add a line to the end of a file by DrHyde (Prior) on Feb 18, 2004 at 10:12 UTC
If you open a file with `O_APPEND` (which is what `>>` does) then each write to the file is preceded with a seek to the current end of the file. The seek and write are atomic, provided that you are writing less than a block of data (might work for more than a block, I disremember). The size of a block varies from one filesystem to another and depends on the platform, the size of the filesystem, the options with which the fs was created, the phase of the moon, and so on. It is typically of the order of 1 to 4 K. Be aware that this race condition does exist in NFS. Which doesn't provide adequate locking to get around it. Yay.	[reply] [d/l] [select]