Hey guys,

I am trying to output logs in Snort and parse them in Perl in real time. I have gotten the parsing code down and the logging from Snort down. Now I am figuring out a way to do both at the same time. Since a log from Snort is similar to this: 

[**] hiiii [**]
06/09-13:41:55.785538 00:1A:A0:D3:BD:62 -> 00:1E:7A:20:C8:83 type:0x80
+0 len:0x1F0
192.17.123.101:49437 -> 131.118.254.38:80 TCP TTL:128 TOS:0x0 ID:25055
+ IpLen:20 DgmLen:482 DF
***AP*** Seq: 0xF6643B04  Ack: 0x7F9ACEDF  Win: 0x4074  TcpLen: 20
47 45 54 20 2F 35 2F 31 34 30 38 2F 31 33 38 38  GET /5/1408/1388
2F 32 30 30 35 31 31 30 34 30 33 2F 31 61 31 61  /2005110403/1a1a
31 61 64 39 34 38 62 65 32 37 38 63 66 66 32 64  1ad948be278cff2d
39 36 30 34 36 61 64 39 30 37 36 38 64 38 34 38  96046ad90768d848
62 34 31 39 34 37 61 61 31 39 38 36 2F 73 61 6D  b41947aa1986/sam
70 6C 65 5F 69 54 75 6E 65 73 2E 6D 6F 76 2E 7A  ple_iTunes.mov.z
69 70 20 48 54 54 50 2F 31 2E 31 0D 0A 41 63 63  ip HTTP/1.1..Acc
65 70 74 3A 20 69 6D 61 67 65 2F 6A 70 65 67 2C  ept: image/jpeg,
20 69 6D 61 67 65 2F 67 69 66 2C 20 69 6D 61 67   image/gif, imag
65 2F 70 6A 70 65 67 2C 20 2A 2F 2A 0D 0A 52 65  e/pjpeg, */*..Re
66 65 72 65 72 3A 20 68 74 74 70 3A 2F 2F 73 75  ferer: http://su
70 70 6F 72 74 2E 61 70 70 6C 65 2E 63 6F 6D 2F  pport.apple.com/
6B 62 2F 48 54 31 34 32 35 0D 0A 41 63 63 65 70  kb/HT1425..Accep
74 2D 4C 61 6E 67 75 61 67 65 3A 20 65 6E 2D 55  t-Language: en-U
53 0D 0A 55 73 65 72 2D 41 67 65 6E 74 3A 20 4D  S..User-Agent: M
6F 7A 69 6C 6C 61 2F 34 2E 30 20 28 63 6F 6D 70  ozilla/4.0 (comp
61 74 69 62 6C 65 3B 20 4D 53 49 45 20 38 2E 30  atible; MSIE 8.0
3B 20 57 69 6E 64 6F 77 73 20 4E 54 20 36 2E 31  ; Windows NT 6.1
3B 20 57 69 6E 36 34 3B 20 78 36 34 3B 20 54 72  ; Win64; x64; Tr
69 64 65 6E 74 2F 34 2E 30 3B 20 2E 4E 45 54 20  ident/4.0; .NET 
43 4C 52 20 32 2E 30 2E 35 30 37 32 37 3B 20 53  CLR 2.0.50727; S
4C 43 43 32 29 0D 0A 55 41 2D 43 50 55 3A 20 41  LCC2)..UA-CPU: A
4D 44 36 34 0D 0A 41 63 63 65 70 74 2D 45 6E 63  MD64..Accept-Enc
6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 64 65 66  oding: gzip, def
6C 61 74 65 0D 0A 48 6F 73 74 3A 20 61 31 34 30  late..Host: a140
38 2E 67 2E 61 6B 61 6D 61 69 2E 6E 65 74 0D 0A  8.g.akamai.net..
43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 4B 65 65 70  Connection: Keep
2D 41 6C 69 76 65 0D 0A 0D 0A                    -Alive....

=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
+=+=+

[download]

My current Perl code works for an output file consisting of many of these. I read @array = <SNORTFILE> so this would be in an array format. My question is, is there any way instead of using File::Tail in this way:

use File::Tail; $file=File::Tail->new("/some/log/file"); while (defined($line=$file->read)) { print "$line"; }

to read each line, can I read each alert and parse that? Or is there another method I am not thinking of? Reading line by line, I can't figure out a way to say anything after the hiiii and the GET, I should keep.

Hopefully this isn't too confusing. Thanks in advance.

Alvin

Update: This is what I have so far. When I run it, it gives me this output. I dont know why it doesn't write to a file nor gives me the GET string correctly. I have almost the same code running on a text file and it outputs perfectly.

CODE: 

#file to write to 
$writeFile = "C:\\Snort\\log\\test3\\write.log";
open WRITEFILE, ">$writeFile" or die "Could not open write file.";

use File::Tail;
$file= File::Tail -> new ("C:\\Snort\\log\\test3\\logto2.log");
@alert; 

while (defined ($linea = $file->read)){
    if($linea =~m/^\=\+\=/){
        process(@alert);
        @alert = ();
    }else{
        push @alert, $linea;
    }
}

sub process{
    @lines = @_;
    $foundGET = ();
    $foundHTTP = ();
    $foundAccept = (); 
    $counter = 0;
    @locations =();  
    
    #keeps track of where the separate alerts start 
    foreach $line (@lines){ 
        if ($line =~ /hiiii/){
            push (@locations, $counter);  
        }#end if
        $counter++;
    }#end foreach
    
    foreach $number (@locations){
    #for the timestamp
    $timeStamp = substr (@lines[$number+1], 0, 14);
    print "Time: $timeStamp\n"; 
    print WRITEFILE "Time: $timeStamp\n";
    
    #for source IP 
    $thirdLine = @lines[$number+2]; 
    $srcIPColonIndex = index ($thirdLine, ':');
    $sourceIP = substr ($thirdLine, 0, $srcIPColonIndex);
    print "Source IP: $sourceIP\n";
    print WRITEFILE "Source IP: $sourceIP\n";
    
    #for source port 
    $thirdLine = substr($thirdLine, $srcIPColonIndex+1);
    $srcPortSpace = index ($thirdLine, ' '); 
    $sourcePort = substr($thirdLine, 0, $srcPortSpace); 
    print "Source Port: $sourcePort\n";
    print WRITEFILE "Source Port: $sourcePort\n";
    
    # for destination IP 
    $thirdLine = substr($thirdLine, index($thirdLine, '>')+2); 
    $destIPColonIndex = index ($thirdLine, ':');
    $destIP = substr ($thirdLine, 0, $destIPColonIndex); 
    print "Destination IP: $destIP\n";
    print WRITEFILE "Destination IP: $destIP\n";
    
    #for destination port 
    $thirdLine = substr($thirdLine, $destIPColonIndex+1); 
    $destPortSpace = index ($thirdLine, ' '); 
    $destPort = substr($thirdLine, 0, $destPortSpace);
    print "Destination Port: $destPort\n";
    print WRITEFILE "Destination Port: $destPort\n";
    
    #for protocol
    $thirdLine = substr($thirdLine, $destPortSpace+1);
    $protocolSpace = index ($thirdLine, ' '); 
    $protocol = substr($thirdLine, 0, $protocolSpace); 
    print "Protocol: $protocol\n"; 
    print WRITEFILE "Protocol: $protocol\n";
    
    $counterGET=0;
    $counterHTTP=0;
    
    #for GET 
    while ($foundGET=~ ()){
        if(@lines[$counterGET]!~ m/GET/){
            $counterGET++;
        }#end if
        else {
            $foundGET = !();
        }#end else
    }#end while
    
    #for HTTP 
    while ($foundHTTP =~ ()){
        if(@lines[$counterHTTP]!~ m/Accept/ && @lines[$counterHTTP]!~ 
+m/HTTP/){
            $counterHTTP++; 
        }#end if
        else {
            $foundHTTP = !();
        }#end else
    }#end while 
    
    $fullString = "";
    @newArray = (); 
    
    #gets all the data packet lines needed 
    for ($count = $counterGET; $count <= $counterHTTP+1; $count++){
        push @newArray, @lines[$count];
    }#end for
    
    #chops the hex out of the get to http lines
    foreach $line3 (@newArray){
        $line3 = substr ($line3, 49);
    }#end foreach 
    
    chomp (@newArray);
    
    $fullString = join ("", @newArray);
        
    $indexOfAccept = index ($fullString, 'Accept'); 
    $fullString = substr($fullString, 0, $indexOfAccept-2);
    
    print "GET Trace: $fullString\n";
    print WRITEFILE "GET Trace: $fullString\n\n";
    print "\n";
    }#end foreach
}#end sub

[download]

Wrrong output: 

C:\Users\Administrator\Documents>c:\users\administrator\desktop\dummy.
+pl
Time: 06/23-14:34:16
Source IP: 172.17.116.101
Source Port: 49214
Destination IP: 131.118.254.38
Destination Port: 80
Protocol: TCP
GET Trace: 7A:20:C8:83 type:0x800 len:0x1

Time: 06/23-14:34:23
Source IP: 172.17.116.101
Source Port: 49214
Destination IP: 131.118.254.38
Destination Port: 80
Protocol: TCP
GET Trace:

Terminating on signal SIGINT(2)

C:\Users\Administrator\Documents>

[download]

Good output parsing a file that is already made: 

Time: 06/17-10:30:26
Source IP: 172.17.116.101
Source Port: 49279
Destination IP: 65.254.57.211
Destination Port: 80
Protocol: TCP
GET Trace: GET /ad.php?do=html&zid=14678&wd=728&ht=90&target=_top HTTP
+/1.1

Time: 06/17-10:30:27
Source IP: 172.17.116.102
Source Port: 49280
Destination IP: 65.254.57.211
Destination Port: 80
Protocol: TCP
GET Trace: GET /af.php?do=imp&zid=14678&aid=108360&auth=C26DDAD23C&wd=
+728&ht=90&t=1308321036 HTTP/1.1

[download]

If anyone has the time to go over this I would appreciate it. I don't know why my code doesn't write or get the right lines. Notice how the last packet doesnt even finish the GET string part.

In reply to Help with Snort and File::Tail by ahuang14

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.