Thanks a lot!

For your code, you used the greedy match (.*). You will use the non-greedy match (.*?) in your code, you will get all the addresses.

while($lines =~ m/traffic-gen.*?client address = 192\.0\.0\.(\d+).*?da
+ta units received = (\d+).*?jitter/scgi)
{ 
    print $1, "\tyes\t",$2,"\n";
    push @ones, $1;
    push @twos, $2;
}
[download]

Yes that worked!! Thanks :)

In general when processing log files, "slurping" or reading the whole file into a single $var is a bad idea. Usually better is to process the log file one line at a time. One reason is so that your code doesn't depend upon the size of the log file. Another reason is for exactly the problem that you are experiencing, the regex stuff gets more complex. There are also some issues about dealing with corrupted lines or records and such things. But that is not relevant to your current problem.

I don't know where you heard about:
 my $lines = do { local $/; <STATFILE> };, but that uses two relatively rare constructs in the same statement! It is highly likely that you can write Perl code for a number of years and never need either "do" or "local".

Below, I wrote a simple parser for you. There is a predefined handle called DATA which I used instead of opening an external file.

The code loops on each line of DATA and searches for a line that ends with "Client Address = some_ip_address". Then the code calls get_units_rcvd() to get that number and the results are printed. That's it. Well code does loop and do the same thing again!

The subroutine, get_units_rcvd() could be written more compactly as could all of this code. But more compact doesn't mean "faster" and there is no need here. This is a demo of a common methodology. Look for something that "starts the record" and then call a subroutine to complete the job.

#!/usr/bin/perl -w
use strict;

while (<DATA>)
{
    if (m/Client Address = ([0-9.]+)\s*$/)
    {
       my $ip_adr = $1;
       my $units_rcvd = get_units_rcvd();
       print "$ip_adr => $units_rcvd\n";
    }
}

sub get_units_rcvd
{
   while (<DATA>)
   {
      if (/Data Units Received = ([0-9]+)\s*$/)
      {
         return ( $1);
      }
   }
}

=prints:
192.0.0.6 => 260
192.0.0.4 => 351
192.0.0.5 => 207
192.0.0.2 => 401
192.0.0.3 => 340
=cut
[download]

I don't know where you heard about: my $lines = do { local $/; <STATFILE> }; , but that uses two relatively rare constructs in the same statement! It is highly likely that you can write Perl code for a number of years and never need either "do" or "local".

That's a common idiom mentioned in perlfaq5 under the question "How can I read in an entire file all at once?". A Super Search confirms that this idiom is quite popular and well-known.

Yes this idiom albeit well known is often misused. I added some more comments at: Re^3: multiple regexp matches in multi line string.

Thanks a lot!!
I guessed that loading the complete file into a variable will be bad practice...thanks for pointing out the better method. I got this idiom  my $lines = do { local $/; <STATFILE> }; from perlfaq5 but I confess that I did not understand how it works. I just had a vague idea that for using multi-line regexp, there should be multiple lines in the variable. But I am curious to know how it works, especially about "do" and "local".
And also thanks for the tipoff on  <DATA> :)

$/ is called the "input record separator". Every text line that you would read from a text file will contain "\n" at the end and Perl would say "hey that's one 'record'" because $/ is '\n' by default.

my $line_count=1;
while (<FILEHANDLE>)
{
   print $line_count++, " :$_"; 
}
[download]

Above prints a "line number" for each line. And yes there is a special Perl variable that represents that without having to use $line_count. But I figure that Perl is terse enough without needing it here. Above, each thing in $_ will be delimited by the input record separator. I used $line_count to show that there are indeed separate input lines being processed.

If you "undefine" the input record separator from its default of "\n", then the whole file will be read in one single shot! That's because there is no "stopping place" to define what a input record (in this case a "line").

local is a weird critter...it says that in this scope define var $x to be "whatever". In functional terms this is like (when re-defined within a scope): push current value, use this new value, pop previous value when the scope changes.

In previous code,


$/ = undef;
my $lines = <STATFILE>;
[download]

would have the same effect as:


my $lines = do { local $/; <STATFILE> };
[download]

$/'s meaning is not FILEHANDLE specific and if you had multiple files open for read at once, the current value of $/ would apply to any subsequent read from any file, but you only had one file.

This "do {code}" is like a subroutine. You will probably write a lot of code before you really need this. Anyway, after this statement, $/ is auto-magically set back to the default of "\n".

Reading a whole file at a time into memory is good for some situations and is good for records where fields span multiple lines. But you don't have that. You have a record that although it contains multiple input lines, nothing spans a line (a "\n" boundary). This is typical of well designed log file output. Log files often get very large and the ability to process them while only requiring a small subset of lines at a time in memory is often well worth the trouble!

I demonstrated one classic way to process a multi-line record, the technique works in C, JAVA or any language. Perl has some cool other ways. For more reading, check out Flipin good, or a total flop?. Perl has a "range" operator that decides whether you are inside of 2 lines bounded by a regular expression. This allows processing in a similar way to the "classic" method and does not require that the entire log file be read into memory at once...again..often a huge advantage!

Hope this lengthy reply helped. There is a BIG difference between records whose fields span multiple lines and a record that contains a collection of more than one line. Well designed text log file formats do not have the former.