Combine them:

while (chomp($_ = <TOPARSE>)) {
   next unless /\d+\s+(\d+)\s+(1\.2\.3\.4)\s+->\s+(4\.3\.2\.1)\s+(?!TC
+P)([^\s]+)\s+(3456)\s+>\s+(113)/;
   print;
}
[download]

That way, you get the best of both worlds. You just do a negative lookahead (?!) AND a protocol gulper (like ([^\s]+), and you can still search for all your other strings that you want to search for.

This will match on source 1.2.3.4 and destination of 4.3.2.1, but will make sure it is not a TCP packet. Best of both worlds...

Update: Per chromatic's and tye's recommendations, my proposed code would look like this:

while (<TOPARSE>) {
   next unless /^\d+\s+(\d+)\s+(1\.2\.3\.4)\s+->\s+(4\.3\.2\.1)\s+(?!T
+CP\s)([^\s]+)\s+(3456)\s+>\s+(113)/;
   print;
}
[download]

Careful! That construct will not process the last line of certain files. chomp can return a false value.

Now just add a ^ anchor at the front of the regex and you've got a nice solution for the regex (otherwise backtracking will be attempted, wasting time). Note that (?!TCP) only prevents matching against fields that start with "TCP". So you might want (?!TCP\s) instead (and don't chomp).

                - tye

Variables are interpolated into a regex, so you don't need to do a s/THIS/THAT/ on your regex string. Also, its safer if you anchor your regex to the beginning of the string. Here's an example:

my $SOURCE_ADDR = param('source_addr') || '.*?';
my $DEST_ADDR   = param('dest_addr')   || '.*?';
my $PROTO       = param('proto')       || '\w+';
my $SOURCE_PORT = param('source_port') || '\d+';
my $DEST_PORT   = param('dest_port')   || '\d+';
...
my $entry =~ /\A
    \d+\s+
    (\d+)\s+
    ($SOURCE_ADDR)\s+
    ->\s+
    ($DEST_ADDR)\s+
    ($PROTO)\s+
    ($SOURCE_PORT)
    \s+>\s+
    ($DEST_PORT)
/x;
[download]

Note that if a user doesn't input one of the above checks, it defaults to matching any valid input for that field. The IP addresses match against '.*?' because I'm too lazy to do it properly--I don't recommend doing it like that in a real program. Also, you might want to remove any whitespace in the user input. Users will likely put in '! TCP', which won't work the same as '!TCP', which is what you want.

----
I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer

Note: All code is untested, unless otherwise stated

Please use \S+ instead of .*? so that backtracking won't waste CPU time. Please include \s after ($DEST_PORT) so that you won't match just a prefix of the destination port number.

I'd add to the above what perlguy recommended:

my %r= (
    source_addr => '\S+',
    dest_addr   => '\S+',
    proto       => '\S+',
    source_port => '\d+',
    dest_port   => '\d+',
);
for my $param (  keys %r  ) {
    my $value= $cgi->param($param) || "";
    if(  $value =~ s/^!\s*//  ) {
        $r{$param}= "(?!\Q$value\E\s)($r{$param})";
    } elsif(  $value  =~ /\S/  ) {
        $r{$param}= "(\Q$value\E)";
    } else {
        $r{$param}= "($r{$param})";
    }
}

my $regex= qr/
    ^
    \d+\s+
    (\d+)\s+
    $r{source_addr}\s+
    ->\s+
    $r{$dest_addr}\s+
    $r{$proto}\s+
    $r{$source_port}
    \s+>\s+
    $r{$dest_port}\s+
/x;

while(  my $pkt= <DATA>  ) {
    print $pkt   if  $pkt =~ $regex;
}
[download]

                - tye

I'm not known for being one to say "don't use regexes for that" or "Use a database", but in this instance, what you are trying to devise is a Query Language.

You may be able to acheive your aims with regexes, but moving your data into a DB and using SQL will undoubtedly save you considerable time and effort.

the data is generated on the fly, it's a packet capture of traffic on a network segment at any given moment. The data isn't already there... I'm collecting, crunching, and producing output from the data all in one go.

The major pain with trying to select records using regexes is that you have to try and match the whole record instead of just the fields that you are selecting on, hence your difficulties with specifiying the logical select "anything except this". The second problem is that of having your regex match against data in another part of the record than the field that you are interested in.

By imposing some structure on your data--ie. making the fields in the record fixed length--and matching or rejecting on a field-by-field basis rather than trying to match (or not) a whole record at a time, you greatly simplify the process. This is what you would get by moving your data into a flat file DB and using DBI to perform your queries.

At the very least, you should consider fixing the length of the fields of your records. You could then use substr as an lvalue in conjunction with a regex to greatly simplify the process of your queries. Eg.

if (substr($record,  0, 10) =~ $src_ip_of_interest
and substr($record, 10, 10) =~ $dst_ip_of_interest
and substr($record, 20,  4) =~ $proto_of_interest
and substr($record, 24,  6) !~ $src_port_of_disinterest
# etc ...
) {
    #we found a record that matches the query
}
[download]

I think that you can see how much this simplifies the regexes involved. Generating conditionals using this form and using eval to execute them would be much simpler than trying to come up with a generic regex generator.

That said, using BerkleyDB or similar in conjunction with DBI::* would be considerably easier to code and probably much quicker in performance.

---

Examine what is said, not who speaks.
1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
3) Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clarke.

#!/usr/bin/perl

push(@foo, 
   '1 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 80 [SYN]',
   '2 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 113 [SYN]',
   '3 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 123 [SYN]',
   '4 120 2.3.4.5 -> 5.4.3.2 ICMP ? > ? echo (ping) reply',
   '5 120 2.3.4.5 -> 5.4.3.2 ICMP ? > ? echo (ping) request',
   '6 120 2.3.4.5 -> 5.4.3.2 ICMP ? > ? echo (ping) reply',
   '7 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 562 [RST]',
   '8 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 36 [RST]',
   '9 60 1.2.3.4 -> 4.3.2.1 TCP 3456 > 90 [RST]',
);

# assume that when split the fields are as follows..
#line  = '0';
#size  = '1';
#src   = '2';
#dest  = '4';
#proto = '5';
#port  = '8';

# here is what we will test on.. this could be altered to be
# collected via flags, shifted off of ARGV, or passed as 
# params to a CGI easily...
print "proto: ";
chomp(my $i_proto=<>);

print "port: ";
chomp(my $i_port=<>);

# loop over our data set, this could just as easily be a 
# socket or filehandle.. 
for ( @foo ) {

   my @line = split(/\s+/);

   if ($i_proto) {
      (my $tmp = $i_proto) =~ s/^!//;

      if ($i_proto =~ /^!/) {
         next if ($line[5] =~ /$tmp/);

      } else {
         next if ($line[5] !~ /$tmp/);
      }
   }

   if ($i_port) {
      (my $tmp = $i_port) =~ s/^!//;

      if ($i_port =~ /^!/) {
         next if ($line[8] =~ /$tmp/);

      } else {
         next if ($line[8] !~ /$tmp/);
      }
   }

   print "$_\n";
}
[download]

# @data now contains src_addr, dest_addr, proto, and port
   @data = ( split(/\s+/) )[2,4,5,8]

# later test elem 3 instead of 5.. yada yada
[download]

Makeshifts last the longest.

Personally, I'd refrain from using one regex for all your querying -- I find it to hard to read.

Why not just use split() and then use regexes on the individual fields? A sample loop for processing line by line:

$user_param{source_addr} ||= '.'; # default value for param
$user_param{dest_addr} ||= '.';

while (<DATA>) {
    my @data = split(/\s/, $_);
    next unless $data[2] =~ $user_param{source_addr};        
    next unless $data[3] =~ $user_param{dest_addr};
    # ...

    print $_;
}
[download]