edison.pioneer has asked for the wisdom of the Perl Monks concerning the following question:

I'm a novice to PERL and I need to finish writing this for my project in a week. I've written to the following code for Snort rule base.What the program is supposed to do is this. Go to Snort folder in C: drive.In that folder, there will be another folder called "rules", in which there will be files by the extension ".rules". I've to extract only the header part of the file. A rule file can be divided into two types: 1) Rule header and 2) Rule options. I'm interested only in rule header. It begins with the beginning of the file (And most of the times the first word is "Alert") till the first opening bracket is encountered. I want to extract only the header part,convert it to hex (using sprintf and ord)and save it to a file called "camdata.txt".Before including use warnings and use strict,I was getting errors related to scalar found where operator expected. Now I get $rulefile requires explicit package name

use strict; use warnings; # use dignostics; print "RULE_FILE : $rulefile \n"; $rulesdir="C:\\Snort\\rules\\*.rules"; @rulefiles = `ls $rulesdir\/*.rules`; $camfile = "camdata.txt"; # print "RULE_FILE : $rulefile \n"; for $rulefile(@rulefiles){ open(INFILE, "<".$rulefile) or die "Can't open ".$rulefile."\n"; @rules=<INFILE>; close(INFILE); # $headerParts = 0; # why bhave I taken a header part here? # $headerParts++; for($rule =~ /alert/) # $prule = $rule; $prule = substr($prule, 0, index($prule, '(')); print "extracted header : $prule \n"; } # Store header-part # if ($headerParts == 1){ # push(@prules, $prule); }
  • Comment on Scanning multiple files from Snort rules one by one and extracting a particular part to another file - File Handling
  • Download Code

Replies are listed 'Best First'.
Re: Scanning multiple files from Snort rules one by one and extracting a particular part to another file - File Handling
by GrandFather (Saint) on Jul 22, 2015 at 07:05 UTC

    First, well done for using strictures. They are going to help a lot in sorting out problems with your code as you develop it.

    Your immediate issue is you need to add my in front of each variable when you first assign a value to it.

    However, you have a long way to go to get your code complete. I suspect you haven't any chance to complete it within the time you have available. You should talk to your lecturer/tutor about the help you need.

    We can give you some hints, but obviously just providing a solution is not going to help in the long run. Here are a few hints that will help write reliable software using Perl:

    • Use indentation to clearly show nested blocks of code
    • Use three parameter open and lexical file handles: open my $fIn, '<', $ruleFile or die "Can't open $ruleFile: $!\n";
    • Use Perl's built in functions instead of executing command line tools. Perl's functions will work across most operating systems and are generally more robust against changes in the OS and shell.
    Update: fixed missing ',' in sample open code - thanks Laurent_R
    Premature optimization is the root of all job security
Re: Scanning multiple files from Snort rules one by one and extracting a particular part to another file - File Handling
by Laurent_R (Canon) on Jul 22, 2015 at 09:18 UTC
    Hi edison.pioneer,

    Your immediate error:

    print "RULE_FILE : $rulefile \n";
    At this point in the code, the $rulefile has not been declared (which is done with the my built-in), leading to a compile-time error, and has not been defined or initialized (which is done by assigning a value to it), leading to a run-time error once the compile-time error has been fixed.

    Next error (assuming you have added the my function for every new variable):

    my $rulesdir = "C:\\Snort\\rules\\*.rules";
    This does not define a directory, so that the next code line will not work as expected. If you want to define a directory, you may want to have:
    my $rulesdir = "C:\\Snort\\rules\\";
    or perhaps simply:
    my $rulesdir = "C:/Snort/rules/";
    Next issue:
    @rulefiles = `ls $rulesdir\/*.rules`;
    From looking at your directory paths, you're working on Windows. The ls command is a Unix system command, it will not work under Windows. You should use Perl internal commands rather than system calls whenever possible. Try this:
    my $rulesdir = "C:\\Snort\\rules"; my @rulefiles = glob "$ruledir/*.rules";
    Another issue (assuming that you have used my and indented your code correctly, as recommended, and straitening a bit the syntax):
    for my $rulefile (@rulefiles){ open my $INFILE, "<", $rulefile or die "Can't open $rulefile $!"; my @rules = <$INFILE>; close $INFILE; }
    Here, I have corrected and/or improved the syntax, but you have a serious algorithmic problem: you are looping over a list of files (@rulefiles) and, each time through the loop, are assigning the contents of the file to @rules, meaning that when you read the content of the second file, you clobber the previous content of @rules (i.e. what you read from the first file), and so on, so that, in fine, you end up with only the content of the last file.

    There are several (actually, many) other issues, but at least, I hope this will help you going forward.

      Laurent_R: ...From looking at your directory paths, you're working on Windows. The ls command is a Unix system command, it will not work under Windows. You should use Perl internal commands rather than system calls whenever possible. Try this: ...

      FWIW, even unix system commands can be available for windows , ls.exe , tar.exe ... not uncommon with perl programmers

        Yes, sure, you're right, it can be available for Windows, but that requires some specific software installation and it is usually not available on a common (or fresh) Windows installation.
      Thanks for pointing out my mistakes!! About ls being a UNIX tool and not Windows. I googled and came across a suggestion on some online forum saying that in environmental variables in PATH I must be adding "C:/Windows/System32" and then reboot. Unsurprisingly, it didn't work!
Re: Scanning multiple files from Snort rules one by one and extracting a particular part to another file - File Handling
by tangent (Parson) on Jul 22, 2015 at 11:16 UTC
    I'm not sure just how big each rule file is, but reading in all of the file seems wasteful when you only want the first part. To carry on with Laurent R's example, you could extract the header part in the first loop and save it into an array for processing later:
    my @headers; for my $rulefile (@rulefiles){ open my $INFILE, "<", $rulefile or die "Can't open $rulefile $!"; my $header; while ( my $line = <$INFILE> ) { # see if this line contains opening bracket if ( $line =~ m/\(/ ) { my $pos = index( $line, '(' ); last unless $pos > 0; $header .= substr( $line, 0, $pos ); last; } else { $header .= $line; } } close $INFILE; push( @headers, $header ) if $header; } for my $header ( @headers ) { print "\nHeader:\n$header\n"; # now process the header }
      Hi, thanks for guiding !! I'm trying to learn this language but I'm always confused in this part. for my $rulefile (@rulefiles){ What exactly is the difference betweeen $rulefile and @rulefiles? I understand that @rulefiles is an array and is actually referring to the assortment of all the ".rule" file in the folder. but what purpose is $rulefile serving?
        Hi,

        @rulefiles is an array containing the names of the *.rule" files in the folder.

        for my $rulefile (@rulefiles) { # ... }
        is a loop that goes over each of the items (file names) in the @rulefiles array, one after the other, assigns it to the $rulefile variable, so that each of the items can be manipulated with the $rulefile variable name within the body of the loop.

        This is a simple example of a Perl one-liner using a similar construct:

        $ perl -e 'for my $i (1, 4, 3, 2) { print "$i\n"; }' 1 4 3 2 $
        In this example, the $i variable takes successively each value of the list (1, 4, 3, 2) and the body of the loop simply prints $i to the screen.
Re: Scanning multiple files from Snort rules one by one and extracting a particular part to another file - File Handling
by Anonymous Monk on Jul 22, 2015 at 07:04 UTC
    So what question is most important to you?