I guess this question somewhat relates to a question I had yesterday ==>
node 446709 <==
I have several large files (about 20 of them) each between 100 and 220 mb in size. The beginning of each line of these files conatains the following. (WkDay Month Day Time Year LogInfo)
Thu Apr 7 03:00:38:81 2005
rest of line is logging info
I am currently searching the files like this:
for $_(sort @voctable)
{
open (TMPFILE, "$workdir$_") or die("unable to open $_: $!\n"),br;
print "[<font color=\"#ff0000\" size=\"+1\">FILE=$_</font>]\n",br;
print hr;
while (<TMPFILE>)
{
print ("<b>$1</b> $2\n",br) if ( m/(.*$MONTH.*$DAY.*$HOUR:$MIN
+UTE:\d\d:\d\d\s+20\d\d)(.*)/g )
}
close (TMPFILE);
}
There is one of these loops for each type of file. There are 5 different types of files. The point of all of this is to grab all lines from like date and time and put them onto one web page, so our application people can have more of an overall view of all logs from the same time period. Currently the process to iterate over each type of file (5), and each type of file can have four or five logs in its history, can take up to 20 minutes. Is there a faster way to do this?
Thanks in advance
Ted
UPDATE
Thanks! Changing the regex did speed things up quite a bit, The regex I am trying now that seems to be a bit speedier is:
print ("<b>$1</b> $2\n",br) if ( m/(^....$MONTH..$DAY.$HOUR:$MINUTE:..
+:...20..)(.*)/ )
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.