SamSmith has asked for the wisdom of the Perl Monks concerning the following question:

How can I merge a number of text files into one in a particular sequence? The content of the files are ~ delimited. The files themselves need to be merged in a particular sequence; lowest first. The script needs to loop through a directory of a number of text files. The content of the files are ~ delimited. For example File1.txt File2.txt File3.txt File4.txt -The script needs to loop through a directory containing these files. -File1.txt is the lowest numbered file and should be merged into file File2.txt as well as removing all duplicates from file two. -This leaves us with a merged File1.txt. -File1.txt should ben be merged with File3.txt as well as removing any duplicates from file three. -File1.txt should then be merged with File4.txt as well as removing any duplicates from file four.. -This leaves us with a merged File1.txt containing unique vales off all four files. -The remaining files should then be deleted.
  • Comment on How can I merge a number of text files into one in a particular sequence?

Replies are listed 'Best First'.
Re: How can I merge a number of text files into one in a particular sequence?
by blindluke (Hermit) on Nov 27, 2014 at 13:19 UTC

    What have you tried so far? Have you tried merging the files first, without removing the duplicates? Have you used Search to find similar questions? Like this: script to merge files

    - Luke

[reply]
Re: How can I merge a number of text files into one in a particular sequence?
by james28909 (Deacon) on Nov 27, 2014 at 16:41 UTC
    I dont mind at all to help someone when i can, but it always helps if you atleast post some example files, and some code that you have tried.

    But just to start it off, this is how I slurp a directories contents usually: 
    use strict;
use warnings;
use diagnostics;

use File::Slurp;

my @files = read_dir($ARGV[0]);

foreach my $file(@files){
    next if (-d $file); #skip directories
    print "processing $file\n";
    ...do stuff...
}

    [download]
    I know your wanting to add everything from file 2 and 3 and 4 to file 1 if it does not exist in file 1, which could be pretty straight forward, but without any example files or code, then I dont know what you really are hoping to end up with in the end.
[reply]
[d/l]
Re: How can I merge a number of text files into one in a particular sequence?
by Anonymous Monk on Nov 27, 2014 at 13:16 UTC

    Using sort (GNU coreutils) 8.4 ... 

    sh
sort -u -t '~' file-1 file-2 file-3 file-4 > x \
&& mv -f x file-1

    [download]
[reply]
[d/l]

      ... oh, the clean up ... 

      rm -f file-2 file-3 file-4

      [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom