in reply to Re^3: UNIX command - remove 0 byte file
in thread UNIX command - remove 0 byte file

A potential problem with using xargs is that although it means only one execution of rm which bodes better performance, it also converts everything read in from the pipe into a single argument list

No, it doesn't. xargs is smart enough that it knows the system limits, and splits of the task into several processes if needed. That's the beauty of xargs. 

$ find /opt/perl -print0 2> /dev/null | xargs -0 | wc -l
     42
$

[download]
Split up into 42 calls to /bin/echo (the default for xargs). Note also the use of -print0 and -0, that eliminates problems with filenames containing whitespace.

Replies are listed 'Best First'.
Re^5: UNIX command - remove 0 byte file
by thor (Priest) on Sep 23, 2005 at 13:28 UTC
    Note also the use of -print0 and -0, that eliminates problems with filenames containing whitespace.
    I think that those are Linux-only features as well. I sure like them when I can use them, though. :)

    thor

    Feel the white light, the light within
    Be your own disciple, fan the sparks of will
    For all of us waiting, your kingdom will come

[reply]
      Actually, they are not Linux-only feature. At worst, they are GNU-only features, and the GNU file/shell utilities happely run on all commonly used Unix platforms. And they work on Windows as well.
[reply]
        You're right, of course. However, often you won't get GNU utils on a system where the vendor packaged versions of those same utils work just fine.

        thor

        Feel the white light, the light within
        Be your own disciple, fan the sparks of will
        For all of us waiting, your kingdom will come

[reply]
Re^5: UNIX command - remove 0 byte file
by Moron (Curate) on Sep 23, 2005 at 13:59 UTC
    Let's not forget the OP specifically said 'unix' rather than linux. Filenames with whitespace not being handled with xargs is certainly going to be a problem if this is supposed to be a generic solution. But if the whole file is enclosed in quotes when it has whitespace, rm will handle the argument list properly even if it went thru xargs: 
    find /path/ -size 0 | perl -e 'while(<>) { chop; s/^(.*\s.*)$/\"$1\"/;
+ print "$_\n"; }' | xargs -n <whatever-filecount-limit> rm

    [download]

    -M

    Free your mind

[reply]
[d/l]
      Except that you now have problems if the filename contains double quotes. Or backquotes. Or dollars. If you want to do the quoting yourself, better wrap the arguments in single quotes, after first escaping any single quotes: 
          find /path/ -size 0 |
    perl -ple 's/\x27/\x27"\x27"\x27/g; "\x27$_\x27"' |
    xargs -n ... rm

      [download]
      Note the use of \x27 for single quote, lest it ends the argument to perl.

      Of course, if you're going to pipe into perl, there's not much point in calling xargs rm is there? Might as well do the removal from within perl: 

          find /path/ -size 0 | perl -nle 'unlink or warn "unlink $_: $!\n"'

      [download]
      Note that both pieces of code given above still fail on filenames containing newlines.
[reply]
[d/l]
[select]
        the whole point of using xargs in this discussion subthread was to get unix to process the list of removals in one go rather than per file. I am not sure if the same effect can be achieved from perl - unlink will accept multiple arguments (so you could push to array before passing the array to unlink), but I don't know if any optimisation is achieved that way.

        -M

        Free your mind

[reply]
Re^5: UNIX command - remove 0 byte file
by Moron (Curate) on Sep 23, 2005 at 13:06 UTC
    I looked into this for xargs under Solaris and the manual makes no such claim, although it looks like you can do something similar manually using a combination of -l -s and -n options, assuming you know the system limits to target these parameters for.

    -M

    Free your mind

[reply]

In Section Seekers of Perl Wisdom