Sorry, I didn't include log entries with 206's in them, but that isn't important here. Notice that I get the local file size by running this perl script in the directory where the downloaded files are located. That's how I calculate how many total bytes have been downloaded through the webserver.
  1. Stat local file
  2. Get local filename from file on disk
  3. Parse Squid logs for entries which match that filename
  4. Multiply number of entries in the logs for that filename by number of bytes that the file-on-disk occupies

What I need to do, I think, is compare the file size of the local file, with the file size value in the log entry, and if it matches, count it as a "completed" download. If not, ignore it. I'm not sure this is accurate either though, because some downloaders can "resume" partial downloads.

Another thought springs to mind though, what if I just concatenate the byte size value in the log itself, only, on a per-file basis, so I'm only parsing bytes out of the logs, not bytes from local files? That would at least allow me to see how many bytes the server sent to clients, but now I have to somehow correlate that on a per-file basis, which could require multiple passes through the logs. Not fun.

I'm open to other ideas, if anyone has them.

In reply to Re: Re: File download statistics parsing by Anonymous Monk
in thread File download statistics parsing by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.