in reply to Append big files

I do not think 1Gb is a problem for a Perl oneliner:

perl -ple "BEGIN{open $to,'>',shift @ARGV;select $to}" destination. +txt source1.txt source2.txt

The -p print each line, -l does automatic lines handling; the BEGIN block shift @ARGV using that file as destination, select print everything to the destination.

PS: if you want something that can print to a destination file or to STDOUT you can modify the above in:

perl -ple "BEGIN{open $to,'>',shift @ARGV and select $to if $ARGV[0]}" # to STDOUT perl -ple "BEGIN{open $to,'>',shift @ARGV and select $to if $ARGV[0]}" + 0 src1 src2 # to file perl -ple "BEGIN{open $to,'>',shift @ARGV and select $to if $ARGV[0]}" + dst src1 src2

PPS: maybe this is more intellegible

perl -ple "BEGIN{open $to,'>',shift @ARGV and select $to unless $ARGV[ +0] eq 'STDOUT'}" # to STDOUT perl -ple "BEGIN{open $to,'>',shift @ARGV and select $to unless $ARGV[ +0] eq 'STDOUT'}" STDOUT src1 src2

L*

There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Replies are listed 'Best First'.
Re^2: Append big files -- oneliner
by BrowserUk (Patriarch) on Sep 15, 2016 at 11:58 UTC

    Your one-liners can be simplified to just:perl -pe1 file1 file2 file3 > allfiles

    (Or just perl -pe1 file* > allfiles under *nix),

    but it won't be as fast as your local system utility; and if one of the files doesn't contain any newlines, it will get very slow indeed, as it will slurp the entire file before writing it back to disk.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.