G'day eversuhoshin,

"I have looked up Alarm but I don't know how to incorporate it into my code."

Using the same code structure given in the alarm example, here's how you might do this:

#!/usr/bin/env perl

use strict;
use warnings;

my @iterations = map { 10 ** $_ } 0 .. 10;
my $timeout = 2;    

for my $iterations_this_loop (@iterations) {
    eval {
        local $SIG{ALRM} = sub { die "TIMEOUT: $iterations_this_loop\n
+" };
        alarm $timeout;

        for (0 .. $iterations_this_loop) {
            # Processing here
        }

        alarm 0;
    };

    if ($@) {
        die $@ unless $@ =~ /^TIMEOUT: \d+/;   # propagate unexpected 
+errors
        print $@;
        next;
    }
    else {
        print "ENOUGH TIME: $iterations_this_loop\n";
    }
}
[download]

Output:

$ pm_long_for_alarm.pl
ENOUGH TIME: 1
ENOUGH TIME: 10
ENOUGH TIME: 100
ENOUGH TIME: 1000
ENOUGH TIME: 10000
ENOUGH TIME: 100000
ENOUGH TIME: 1000000
ENOUGH TIME: 10000000
TIMEOUT: 100000000
TIMEOUT: 1000000000
TIMEOUT: 10000000000
[download]

for my $iterations_this_loop (@iterations) { represents the number of records in each file (that will be foreach (@files){ in your code).

for (0 .. $iterations_this_loop) { represents processing that number of records (that will be the processing you are doing within each iteration of your loop).

Note: the argument to alarm is in seconds - for 2 minutes you'll need my $timeout = 120;.

I looked at the first couple of regexes (see below). When you are using the /x modifier, you can space this out on multiple lines and this can improve the readability a lot. You can also add comments to the lines, but there are some limitations about what can go in the #comment (see perlre doc) for more details and you cannot put a space inside of a 2 char token like the ?: in (?: ..the non-capture..), but this #comment stuff can be useful.

I see some strange things (there appear to be terms that have no purpose). Also $data (maybe a 10MB) is slurped into memory as a single variable and many regex'es are applied serially to this humongous thing. Parsing, re-parsing, re-parsing and re-parsing something big is often not a good idea performance wise.

Often, parsing something very large is best done line by line and ONLY once. Read a line, deal with it, throw it away because we are done....

I suspect that if you shared some more details about the file format and why one of these things is 10MB?, far more efficient algorithms could be devised. Your regex'es appear to do very similar things. A single pass that figures everything out on "one go" would be faster. Could even be that algorithms that just stop reading the file, once we've got what we need are appropriate?

While I was playing with this, I spaced your regex'es out (that is what the /x allows). Also show how to use the Regex::Explain function - which is sometimes useful.

So some alternate way to space out the regex'es to increase readability are shown below.

I do suspect that the "real solution" is to make this so fast that there is never any need for a 2 minute timeout! But there are some things about your application that I and others just don't understand. It would be most helpful if you could clarify further!

#!/usr/bin/perl -w
use strict;

use YAPE::Regex::Explain;

#prototype from the docs...
#my $exp = YAPE::Regex::Explain->new($REx)->explain;

my $REx1 = 
    qr{m/(?:Item|ITEM)[Ss]?\s?
    (?:\.|\-|:|\-\-|\,)?\s?
    (?:1|I)\s?
    (?:\.|\-|\:|\-\-|\,)?\s?
    (?:Description|DESCRIPTION)?\s?
    (?:[Oo][Ff])?\s?(?:[Tt][Hh][Ee])?\s?
    (?:Busine\s?ss|BUSINE\s?SS|Company|COMPANY)\s?
    (?:\.|\-|:|\-\-|\,|\()?
    (.*?)\s?
    (?:Item|ITEM)[Ss]?\s?
    (?:\.|\:|\-\-|\-|\,)?\s?
    (?:I|1A|1B|2)\s?
    (?:\.|\:|\-\-|\-|\,)?/x}; #this term apparently 
     # not needed not captured and it is optional
                                                        
my $Rex2 = 
    qr{m/(?:Business\s?Development|BUSINESS\s?DEVELOPMENT)\s?
    (.*?)\s?
    (?:Item|ITEM)[Ss]?\s?
    (?:\.|\-|:|\-\-|\,)?\s?
    (?:I|1A|1B|2)\s?
    (?:\.|\:|\-\-|\-|\,)?/x};
                
my $Rex3 = 
    qr{m/(?:PART|Part)\s?
    (?:\.|\-|\:|\-\-|\,)?\s?
    (?:I|1)\s?
    (?:\.|\-|:|\-\-|\,)?\s?
    (?:BUSINESS|Business|GENERAL|general)
    (.*?)\s?
    (?:Item|ITEM)[Ss]?\s?
    (?:I|1A|1B|2|3)\s?
    (?:\:|\-|\,|\-\-|\.|\,)?/x};                

print YAPE::Regex::Explain->new($REx1)->explain;
[download]

if($data!~m/table\s?of\s?contents?|\sindex\spart\s(?:1|I)/i){
[download]

the end result is just a text file with the file name, excerpt. Thank you for asking :) Cheers, Pureum

But what is input file format? 10MB file is nothing for Perl, so if you could share some sample file (one line is enough) we could look at regexes, I guess those can be optimized in some way.