Suwen has asked for the wisdom of the Perl Monks concerning the following question:

hi friend,

I'm beginner of Perl. I get into the following trouble about a few days. Hope you give me some advice.
I have a logfile like this:

Sat Jul 21 22:30 2001 144.02.26.399 www.myserver.com ...
Sat Jul 21 23:01 2001 352.34.12.345 www.myserver.net ...
........................................................
........................................................
i want to parse it to a HTML table format like this:
Sat Jul 21 22:30 2001 144.02.26.399 www.myserver.com .........
Sat Jul 21 23:01 2001 352.34.12.345 www.myserver.net .........
........ ...... ...... .........

Cheers

Suwen
  • Comment on How to parse a logfile to a HTML table format

Replies are listed 'Best First'.
Re: How to parse a logfile to a HTML table format
by HyperZonk (Friar) on Jul 22, 2001 at 02:28 UTC
    Update: I note that some people are recommending substr solutions. If your log is like mine, it isn't actually fixed width fields (though it may appear to be at first, if all of the IP addresses have the same width). There is, of course, a way to work with that (using index), but that probably is less efficient. Personally, my favorite answer from below is the split answer, which is probably better than my regex solution.

    I'm going to be very general in my answer, to hopefully increase the usefulness of the answer (and salve my conscience about being a homework-troll).

    Say you've got line-length records of the form:
    xxx yyy zzz aaaaa bbb cccccccccccc...
    Say you want x, y, and z in one field, and the rest of the stuff in separate fields. You can populate an array (@infile) with the lines from the log file, then do the following:
    foreach (@infile) { /(\S+\s+\S+\s+\S+)\s+(\S+)\s+(\S+)\s+(.*)/; my @fields = ($1,$2,$3,$4); push @output, \@fields; }
    This will create a list of lists that you can output as you need. @output is a list of references to anonymous lists populated with your desire output fields. \s parses whitespace characters, \S parses non-whitespace character. You simply arrange your parentheses to capture however many of the space delimited items you need per field.

    Note: this code could be optimized for shortness, but I have tried to keep what is happening relatively obvious by explicitly doing things in several steps.

      A short remark, you suggest to

      populate an array (@infile) with the lines from the log file.
      This is absolutely not necessary in this case, it's just a waste of memory. Why not do it like this:
      open(LOG,'<','logfile') or die "Couldn't open logfile: $!"; while (<LOG>) { # do something } close LOG;

      -- Hofmator

Re: How to parse a logfile to a HTML table format
by abstracts (Hermit) on Jul 22, 2001 at 03:26 UTC
    Hello,

    I assume you know how to use the function split. Type "perldoc -f split" if you don't. Using split, you can get all fields and join the fields you want to produce the strings required.

    Now from looking at your logs, it seems like all columns have fixed widths. The first 21 chars go to the first cell in the table. Skip one field, then grab the next 13 chars, skip one and get the rest.

    To match these fields, one can use a regular expression to do the job. /^(.{21}) (.{13}) (.*)/ should match 21 chars, followed by a space, followed by 13 chars, followed by a space, then the rest. In list context, the matching operator returns a list of all things between brackets.

    # example:
    my $str = "Sat Jul 21 22:30 2001 144.02.26.399 www.myserver.com";
    my ($date, $stuff, $servers) = $str =~ /^(.{21}) (.{13}) (.*)/;
    
    So, to accomplish your task, just do a simple:
    print "<table>";
    for(@lines){
        print "<tr>";
        print "<td>$_</td>" for /^(.{21}) (.{13}) (.*)/;
        print "</tr>";
    }
    print "</table>";
    
    And you should be done.

    Hope this helps,,,

    Aziz,,,

Re: How to parse a logfile to a HTML table format
by tachyon (Chancellor) on Jul 22, 2001 at 03:28 UTC

    print "<table border='3'>\n"; while (<DATA>) { next if m/^\s*$/; my @elements = split /\s+/, $_; print <<" LINE"; <tr> <td>@elements[0..4]</td> <td>$elements[5]</td> <td>$elements[6]</td> </tr> LINE } print "</table>"; __DATA__ Sat Jul 21 22:30 2001 144.02.26.399 www.myserver.com ... Sat Jul 21 23:01 2001 352.34.12.345 www.myserver.net

    I read from the DATA filehandle for this demo. You will need to open your logfile to a filehandle to read from it. You do that like this:

    print "<table border='3'>\n"; open FILE "</path/to/logfile.txt" of die "Oops $!\n"; while (<FILE>) { next if m/^\s*$/; my @elements = split /\s+/, $_; print <<" LINE"; <tr> <td>@elements[0..4]</td> <td>$elements[5]</td> <td>$elements[6]</td> </tr> LINE } print "</table>"; close FILE;

    Normally I would also post some explanation but as you have not posted any code I will leave it up to you to research anything in this code that is unfamiliar. You will need to add the usual HTML HEAD and BODY tags to complete a valid page.

    cheers

    tachyon

    s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

Re: How to parse a logfile to a HTML table format
by ChemBoy (Priest) on Jul 22, 2001 at 04:17 UTC

    Others have suggested various methods for parsing your logfile, to which I would add a suggestion to try looking at substr, especially if you're not yet totally comfortable with regular expressions and the uses thereof (and assuming that your records are fixed-width).

    You don't say if you're planning to generate this table dynamically or to store it: if you're doing it on the fly, then I hope you're using CGI.pm, and if you are I suggest outputting the table the easy way, using the Tr and td functions.

    If you have a row of data in an array variable (say, @foo), you can generate table cells for that row with the statement

    my $html = td (\@foo);
    and the Tr function can be used similarly.

    So the relevant section of your code could look something like this highly off-the-cuff and untested fragment:

    my @rows; while (<LOGFILE>) { my @data; #your routine of choice to split the input into fields #and stash it in @data push @rows, td(\@data); } print table (Tr (\@rows));

    Which, as you see, makes the whole formatting question pretty much go away.

    If you're not doing this dynamically, you could still use the CGI module (use CGI qw (:all -nodebug), but there are probably simpler solutions available on CPAN (try looking under the WWW section for HTML building solutions).



    If God had meant us to fly, he would *never* have give us the railroads.
        --Michael Flanders

Re: How to parse a logfile to a HTML table format
by Suwen (Initiate) on Jul 23, 2001 at 00:21 UTC
    Thank you very mcuh.

    Suwen