solri1 has asked for the wisdom of the Perl Monks concerning the following question:

I'm using Text::Balanced for an implementation of template like system. I've gotten it to work exactly as it's supposed to, except it's horribly slow. I'm wondering if I'm doing something wrong. The idea is that given some text marked up like so: 
This is text 0.
<SECTION[1]>
This is text 1.
</SECTION>
This is text 0.
<SECTION[2]>
This is text 2.
</SECTION>

[download]
We specify which section tags to show and which to delete. So, if I called this function on the above text with only section 1 set to show, I'd get: 
This is text 0.
This is text 1.
This is text 0.

[download]
Here's my implementation/test program: 
#!/usr/bin/perl -d:DProf

use Carp;
use Text::Balanced;

our $extract_section_tag  =  Text::Balanced::gen_extract_tagged( '<SEC
+TION\[([^\]]+)\]>\n?', '</SECTION>\n?', '[\S\s]*?(?=<SECTION\[)' );

my $page;
$page .= "This is some filler.\n" x 20000;
$page .= "<SECTION[test1]>\n";
$page .= "This is some filler.\n" x 20000;
$page .= "</SECTION>\n";
$page .= "This is some filler.\n" x 20000;
$page .= "<SECTION[test2]>\n";
$page .= "This is some filler.\n" x 20000;
$page .= "</SECTION>\n";
$page .= "This is some filler.\n" x 20000;

print "Calling process_section.  Length of page=[" . length($page) ."]
+\n";
my $newpage = process_section($page, {test1 => 1});
print "Done.  Length of newpage=[" . length($newpage) ."]\n";

sub process_section {
    my ($page, $hashref) = @_;

    my ($tag_section, $post, $pre, $tag_open, $content, $tag_close, @i
+nfo);

    my $return = '';

    while ( @info = $extract_section_tag->($page) ) {
        ($tag_section, $post, $pre, $tag_open, $content, $tag_close) =
+ @info;

        if (! (defined $tag_section && length $tag_section) ) {
            if ($post =~ m/<SECTION\[/) {
                my $excerpt = substr($post, 0, 100);
                print STDERR "\n";
                Carp::carp("Warning: Unbalanced SECTION tags.  Fix the
+ template! Error near: $excerpt\n");
            }
            last;
        }

        my $show = 0;
        if ($tag_open =~ m/<SECTION\[(.*?)\]>/) {
            $show = 1 if (exists $hashref->{$1});
        }

        if ($show && $content =~ m/<SECTION/) {
            $content = process_section($content, $hashref);
        }

        $return .= $pre . ($show ? $content : '');
        $page = $post;
    }

    $return .= $post;
    return $return;
}

[download]
The above code takes over 2 seconds to run on my system, and it gets even worse as the page it's processing gets larger. How can I improve the speed of this? I'm using Text::Balanced v2.0.0, which fixes the issue of a $& variable slowing down all regexps.

Replies are listed 'Best First'.
Re: Am I using Text::Balanced correctly? Speed issues.
by renodino (Curate) on Aug 31, 2007 at 20:16 UTC
    I can't provide a precise answer, but that recursion passing $page by value is probably a good place to start looking. But more importantly, if in fact your task is a simple as stated, why not just use a progressive regex ? Here's my quick hack at it, and it runs quite quickly (Note I haven't actually validated the output string, other than noting the length change reported at exit): 
    use Carp;

use strict;
use warnings;

my $page = join('',
"This is some filler.\n" x 20000,
"<SECTION[test1]>\n",
"This is some filler.\n" x 20000,
"</SECTION>\n",
"This is some filler.\n" x 20000,
"<SECTION[test2]>\n",
"This is some filler.\n" x 20000,
"</SECTION>\n",
"This is some filler.\n" x 20000);

print "Calling process_section.  Length of page=[" . length($page) ."]
+\n";
my $newpage = process_section(\$page, {test1 => 1});
print "Done.  Length of newpage=[" . length($newpage) ."]\n";

sub process_section {
    my ($page, $hashref) = @_;

    my $return = '';
    
    while ($$page=~/\G(.*?)<SECTION\[([^\]]+)\]>\n?/igcs) {
        $return .= $1;
        my $tag = $2;
        my $post = pos($$page);
        if ($$page=~/\G(.*?)<\/SECTION>\n?/igcs) {
            $return .= $1 
                if exists $hashref->{$tag};
            next;
        }

        my $excerpt = substr($$page, $post, 100);
        print STDERR "\n";
        Carp::carp("Warning: Unbalanced SECTION tags.  Fix the templat
+e! Error near: $excerpt\n");
        pos($$page) = $post;
    }

    $return .= substr($$page, pos($$page))
        if (pos($$page) < length($$page));
    return $return;
}

    [download]
    Note that I've changed to use a ref to the $page, and eliminated recursion.

    Text::Balanced is a great tool for dealing with complex things like parsing Perl code, but if you can get by with a good old RE, I'd say its probably a much better solution.

    Perl Contrarian & SQL fanboy
[reply]
[d/l]

Back to Seekers of Perl Wisdom