Certainly, in Unix, I could do something like:

1) Concatenate files into a single file

2) Then do: `sort -u <concatfile> > <sortedfile>`

... but I suspect this will eventually live on a Windows box.

Get your self the sort program from UnxUtils. It supports -u which appears to be all you require to use your unix solution on a windows box:

c:\>u:sort --help
Usage: u:sort [OPTION]... [FILE]...
Write sorted concatenation of all FILE(s) to standard output.

Ordering options:

Mandatory arguments to long options are mandatory for short options to
+o.
  -b, --ignore-leading-blanks ignore leading blanks
  -d, --dictionary-order      consider only blanks and alphanumeric ch
+aracters
  -f, --ignore-case           fold lower case to upper case characters
  -g, --general-numeric-sort  compare according to general numerical v
+alue
  -i, --ignore-nonprinting    consider only printable characters
  -M, --month-sort            compare (unknown) < `JAN' < ... < `DEC'
  -n, --numeric-sort          compare according to string numerical va
+lue
  -r, --reverse               reverse the result of comparisons

Other options:

  -c, --check               check whether input is sorted; do not sort
  -k, --key=POS1[,POS2]     start a key at POS1, end it at POS 2 (orig
+in 1)
  -m, --merge               merge already sorted files; do not sort
  -o, --output=FILE         write result to FILE instead of standard o
+utput
  -s, --stable              stabilize sort by disabling last-resort co
+mparison
  -S, --buffer-size=SIZE    use SIZE for main memory buffer
  -t, --field-separator=SEP use SEP instead of non- to whitespace tran
+sition
  -T, --temporary-directory=DIR  use DIR for temporaries, not $TMPDIR 
+or c:/temp
                              multiple options specify multiple direct
+ories
  -u, --unique              with -c: check for strict ordering
                              otherwise: output only the first of an e
+qual run
  -z, --zero-terminated     end lines with 0 byte, not newline
      --help     display this help and exit
      --version  output version information and exit

POS is F[.C][OPTS], where F is the field number and C the character po
+sition
in the field.  OPTS is one or more single-letter ordering options, whi
+ch
override global ordering options for that key.  If no key is given, us
+e the
entire line as the key.

SIZE may be followed by the following multiplicative suffixes:
% 1% of memory, b 1, K 1024 (default), and so on for M, G, T, P, E, Z,
+ Y.

With no FILE, or when FILE is -, read standard input.

*** WARNING ***
The locale specified by the environment affects sort order.
Set LC_ALL=C to get the traditional sort order that uses
native byte values.

Report bugs to <bug-textutils@gnu.org>.
[download]

Point well taken, TYVM.

Now what's the Perlish way to solve this?

Where do you want *them* to go today?

On the basis of what you've said about the data, it could be as simple as this:

#! perl -slw
use strict;

my $dir = $ARGV[ 0 ] || die 'Need a directory';

my %hash;
while( my $file = <"$dir/*.log"> ) {
    open my $fh, '<', $file or die "$file : $!";
    while( <$fh> ) {
        $hash{ $_ } = 1;
    }
    close $fh;
}

open my $fh, '>', "$dir/composite.log" or die $!;
print $fh $_ for sort keys %hash;
close $fh;
[download]

This assumes that all 31 log files from a particular server are located in a single directory, no other files are in that directory, and that the lines can be sorted using an alphanumeric sort. Eg. Each line carries a date/time stamp at the beginning of the line, and it is ordered in some sensible form (YYYYMMDD HH:MM:SS) that will sort correctly.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Having hit this just last night I thought I'd ask: how long has UnxUtils actually been unavailable? Clicking either of the .zip links gets you "You don't have permission to access /UnxUpdates.zip on this server."

I'd love to equip my brethren des fenêtres with some basic tools (like 'wc') but don't want to ask them to do Cygwin.

Hmm. I hadn't realised that the links didn't work. I just searched for them, but didn't check them as I have had my copies for years. And a quick browse doesn't turn up any reason either?

If you want a copy, /msg me an email address and I'll forward it to you.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Perhaps you could simply keep track of the file position of the last log entry you've processed, and then process all entries found after that. If the file position is smaller than the previous day, then take all the lines. Something like (untested top-of-the-head) this:

# Read logfile names with last position read yesterday
open $inf, "<", "loglist.txt" or die;
while (<$inf>) {
   chomp;
   my ($fname, $fpos) = split /\|/;
   $logs{$fname} = $fpos;
}
close $inf;

# Get new lines from each file
for my $fname (keys %logs) {
   open $ouf, '>>', $fname . ".cumulative" or die;
   open $inf, '<', $fname or die;
   if ($logs{$fname} < stat($inf)[7]) {
      # Continue from where we left off yesterday
      seek $inf, $logs{$fname}, SEEK_SET;
   }
   else {
      # start at beginning of file
   }
   while (<$inf>) {
      print $ouf $_;
   }
   $logs{$fname} = tell $inf;
   close $inf;
   close $ouf;
}

# Rewrite list of files and positions
open $ouf, ">", "loglist.txt" or die;
print $ouf join "\n", map { $_ . '|' . $logs{$_} } keys %logs;
close $ouf;
[download]

Doesn't that leave a hole in the logic where today's file was truncated, but grew to be larger than yesterday's file?

For example a busy Monday after a quiet Sunday?

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

BrowserUk:

Good catch! I guess we'd have to add some code to cache the first line as well. If the first line is the same, do the same thing as above. If not, then cache the first line and take the whole file.

--roboticus