centrum has asked for the wisdom of the Perl Monks concerning the following question:

I use simple script that keeps a log of everyone who "hits" the page, keeping the information in one file. For some reason, it produce unformatted log, without spaces and paragrafs, which is very difficult to browse:

Time: 05/13/05 08:09:12 EDT User: Host: Addr: 22.85.112.194 With: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Page: From: Time: 05/13/05 08:09:18 EDT User: Host: Addr: 22.85.112.194 With: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Page: From: Time: 05/13/05 08:09:28 EDT User: Host: Addr: 22.85.112.194 With: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Page: From: Time: 05/13/05 08:09:31 EDT User: Host: Addr: 22.85.112.194 With: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Page: From:

How to make text separated a little, like this:

Date: Fri May 13 04:00:08 2005
Page Visited: http://www.appt.org.uk/technical/calc_and_convert.htm
Host Name:
Visitor IP: 65.117.182.58
Browser: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050225 Firefox/1.0.1

Date: Fri May 13 04:00:44 2005
Page Visited: http://www.appt.org.uk/technical/size_chart.htm
Host Name:
Visitor IP: 65.117.182.58
Browser: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.6) Gecko/20050225 Firefox/1.0.1
#!/usr/bin/perl # logger.cgi # version 1.0 # Create a file called main.log. Must have write permissions # set to main.log $mainlog = '/path/to/file/stats.txt'; $shortdate = `date +"%D %T %Z"`; chop ($shortdate); print "Content-type: text/plain\n\n "; open (MAINLOG, ">>$mainlog"); print MAINLOG "Time: $shortdate\n"; print MAINLOG "User: $ENV{'REMOTE_IDENT'}\n"; print MAINLOG "Host: $ENV{'REMOTE_HOST'}\n"; print MAINLOG "Addr: $ENV{'REMOTE_ADDR'}\n"; print MAINLOG "With: $ENV{'HTTP_USER_AGENT'}\n"; print MAINLOG "Page: $ENV{'DOCUMENT_URI'}\n"; print MAINLOG "From: $ENV{'HTTP_REFERER'}\n\n"; close (MAINLOG); exit;

Replies are listed 'Best First'.
Re: log file formatting
by Nitrox (Chaplain) on May 13, 2005 at 15:42 UTC
    I'm guessing that you're viewing the log under Windows?

    Print a carriage return along with your new line characters.

    But then I notice that you're also printing a content type header leading me to assume that you are viewing the log through a browser?

    In that case, print the proper HTML line breaks for line endings. -Nitrox

      Proper HTML line breaks wouldn't do much good, the content-type header is text/plain, not text/html.

      He's using a content-type, but it's 'text/plain'. HTML tags would be a bad thing to do, if the file isn't going to be interpreted as HTML. (text/html, text/xhtml, text/xhtml+xml, etc)

      The problem is, he's given us the code for what generates the log, and not for what displays it. I think you're right, in that it's being displayed as HTML, and not using HTML line endings. The easiest solution is to make sure that the browser is sending the correct mime headers when viewing the logs.

      I'd probably also use common+ log format (aka. NCSA extended log format) , as it would allow you to use existing log analysis packages (eg, analog) for those pretty graphs that management likes

      (well, I'd use the format, but I'd do it in the server, rather than generating a new process just to log)

      Update: follow the link that dorward gave for the correct xhtml mime type

        text/xhtml and text/xhtml+xml are not real MIME types. XHTML documents should be served as application/xhtml+xml. See XHTML Media Types for the full story.
      Could you show in my code how must be this proper HTML line breaks for line endings?
      Thanks.
Re: log file formatting
by radiantmatrix (Parson) on May 13, 2005 at 15:46 UTC

    I'm not entirely clear on what you're asking, but I can make a good guess. I can also make a good guess as to what the problem is: you are viewing logs on Windows.

    The "\n" char has different meanings on different platforms. If it is using Unix line endings (as I suspect it is), they would appear as empty space or possibly a small rectangle in Notepad. The easiest solution is to use a Windows editor that will automatically interpret those line endings and display them the way you expect.

    My personal recommendation is SciTE, which is free and pretty nice. UltraEdit also works very well, though it is shareware.


    The Eightfold Path: 'use warnings;', 'use strict;', 'use diagnostics;', perltidy, CGI or CGI::Simple, try the CPAN first, big modules and small scripts, test first.

      Just fyi, notepad may not be able to handle unix lineends, wordpad does it just fine, so no reason to download a program just to view a file. UltraEdit is one fine piece of software when it comes to the more advanced stuff though.

      Remember rule one...

        I NOT INTENDED download files at all! All I need is view log.txt via MSIE web browser. I want it be formatted slightly --just separated as in my example.
      Yes, I use ms Windows, IE6.0 browser to view logs. I view logs via browser, just type path to log file http://www. domain.com/logs.txt This is most convenient for me, no other methods required. What should be corrected in code and how?
      Thanks.
        Well, hold on there, friend -- no need to get punchy. You might have mentioned that you had this problem in your browser, and not in an editor: this is why it's important to be complete when asking a question, here.

        To fix this code-side rather than client-side, you need to instruct your code to use Windows-style line endings. In your code, wherever you see "\n", use "\r\n" instead. If there are a great number of places that you've already used "\n" and you don't want to hunt them down, you could also use a short Perl script to convert them right before viewing. This might look like:

        #!/usr/bin/perl use strict; use CGI::Simple; my $q = new CGI::Simple; $| = 1; print $q->header( -type => 'text/plain', -expires => '+1m' ); ## the header is for plain text, and the data expires in 1m ## If there's no file parameter, exit with an error message unless ($q->param('file')) { print 'No File was requested'; exit; } open LOG, '<', $q->param('file') or do { # exit with error if we can't open the file print 'Unable to open file ',$q->param('file'),':',$!; exit; }; while (<LOG>) { s{(\r\n)|(\n)}{\r\n}g; ## replace unix with Win line-endings print $_; }
        This is untested

        This should be invoked as http://www.domain.com/myname.pl?file=logs.txt (if you named your script 'myname.pl').


        The Eightfold Path: 'use warnings;', 'use strict;', 'use diagnostics;', perltidy, CGI or CGI::Simple, try the CPAN first, big modules and small scripts, test first.

Re: log file formatting
by jfroebe (Parson) on May 13, 2005 at 15:46 UTC

    Unlike notepad, WordPad and Microsoft Word works just fine with unix style text files. There really isn't any need to have two different files.

    Jason L. Froebe

    Team Sybase member

    No one has seen what you have seen, and until that happens, we're all going to think that you're nuts. - Jack O'Neil, Stargate SG-1

      So, there is no workaround get normally formatted log files txt via web browser at all?
Re: log file formatting
by xorl (Deacon) on May 13, 2005 at 15:42 UTC

    What are you using to view the file?

    If it a regular text editor (vi, xemacs, or similar) then it should be seeing the newline (\n) that your script is writting.

    If it an html viewer or something then you'll need to put the proper code for a new line. For an html viewer that would be the BR tag.

    As mentioned above, if heaven forbid you're on an evil OS you might need to use \r or even \r\n instead of \n