Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I run a chatterbox on the allpoetry.com website. We allow formatting for *bold* to be turned bold, etc. We also have smilies, which are added first. The slashes turn out to be the biggest problems, since they're in images often.

So given this text:  go/to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> /thx/ <img src='http://allpoetry.com:8080/images/smile/happy.gif'> silly *to*

Should return: g<em>to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> </em>thx/ <img src='http://allpoetry.com:8080/images/smile/happy.gif'> silly <b>to</b>

Its rather key to NOT do anything to the last '/' - the problem is matching a '/' from the img instead.

As an additional complication, I'd like it to handle nested formatting as well (*/test/* = test)

I've been working on this for a *long* time, with many different iterations , but none seem that close. I'd like to avoid parsing it out into html tags and non-tags, but if anyone can suggest an efficient way to do that I'd love to hear it.

Here is what I have:

sub reg_fix { my $c = shift; $c =~ s#(<.*?>)|(/|\*|_|=)((?:<.*?>|[^\2])+?)\2\s?# if ($1) {$1} elsif ($2 eq '/') {'<em>' . reg_fix($3) . '</em> +'} elsif ($2 eq '*') {'<b>' . reg_fix($3) . '</b> '} elsif ($2 eq '_') {'<u>' . reg_fix($3) . '</u> '}#gemio; return $c; } $c = reg_fix($c);

Which now fails because in the second part: ((?:<.*?>|[^\2])+?)\2\s?#, if it DOESN'T find \2, will backtrack and dip inside the <.*?> tag, which is what I want to avoid. I thought of trying to use (\2|$) at the end, to see if we were at the end, but then it won't finish parsing the string for other matches.

$c =~ s#((<.*?>)|(/|\*|_|=)((?:<.*?>|[^\2])+?)(\2|$)\s?)# sub reg_fix { my $c = shift; print "Running check on '$c'\n"; $c =~ s#((<.*?>)|(/|\*|_|=)((?:<.*?>|[^\2])+?)(\2|$)\s?)# if (!$5) {$1} elsif ($2) {$1} elsif ($3 eq '/') {'<em>' . reg_fix($4) . '</em> '} elsif ($3 eq '*') {'<b>' . reg_fix($4) . '</b> '} elsif ($3 eq '_') {'<u>' . reg_fix($4) . '</u> '}#gemio; return $c; }

Any ideas, great and powerful perl monks? Thanks :)

Replies are listed 'Best First'.
Re: match *bold* formatting, but avoid html (tokenize)
by tye (Sage) on Sep 26, 2003 at 05:08 UTC

    I have fuzzy memory about backreferences in character classes. I recall hearing that they work in quite new versions of Perl and then being straightened out that they don't work and likely never will. That is, that [^\1] always matches any character but "\1" (CTRL-A, "\cA"). You seem to think that [^\2] works and I'm not trying to say you are wrong. Just that you might be wrong.

    So, instead of [^\2] you might want (?!\2) (which wouldn't be the same if $2 were more than a single character).

    Also, <[^<>]+> is better, otherwise:

    Well, if < _$20_, you *need* <a href="...">this</a>
    won't expand either of the _ and * formatting. (Below I took the liberty of turning such unmatched <s into &lt; as well.)

    In a situation like this, I'd "tokenize" rather than "match":

    use warnings; use strict; sub reg_fix { my $text = shift(@_); # I guessed what you wanted =equals= to indicate: my %tag= qw( / em * b _ u = strike ); my %open; @open{ qw( / * _ = ) }= (); my @tokens= $text =~ m{ ( <[^<>]+> | [/*_=] | [^</*_=]+ | < ) }gmx; my @nested; for my $token ( @tokens ) { if( "<" eq $token ) { $token= "&lt;"; } elsif( exists $open{$token} ) { if( ! defined $open{$token} ) { $open{$token}= \$token; push @nested, $token; } else { ${$open{$token}}= "<$tag{$token}>"; my $nest; do { $nest= pop @nested; undef $open{$nest}; } while( $nest ne $token ); $token= "</$tag{$token}>"; } } } return join '', @tokens; } print reg_fix($_) for <DATA>; __END__ go/to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> /th +x/ <img src='http://allpoetry.com:8080/images/smile/happy.gif'> _sill +y *to_* Well, if < _$20_, you *need* <a href="...">this</a>
    which produces:
    go<em>to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> +</em>thx/ <img src='http://allpoetry.com:8080/images/smile/happy.gif' +> <u>silly *to</u>* Well, if &lt; <u>$20</u>, you <b>need</b> <a href="...">this</a>

    Note how I prevent mis-nesting of attributes.

                    - tye
Re: match *bold* formatting, but avoid html
by Fletch (Bishop) on Sep 26, 2003 at 01:28 UTC

    You might see if something like Text::WikiFormat does what you want. Or if not it might give you something similar from which to work.

Re: match *bold* formatting, but avoid html
by delirium (Chaplain) on Sep 27, 2003 at 21:41 UTC
    My approach does split out the lines into markup/non-markup, which you said you want to avoid. I use a simple hash to remember whether it was time for an open or close tag, or no tag at all to prevent opening a tag and not closing it.

    It handles nested tags (e.g., */stuff/*), but doesn't check for mis-nested tags like Tye's solution does.

    Enjoy.

    #!/usr/bin/perl -w use strict; my %sc = qw ( \\* b / em _ u ); my %counts; my %opened; sub track { $opened{$_} = 1 - ( $opened{$_} || 0 ); return "<$sc{$_}>" if --$counts{$_} > 0 && $opened{$_}; return "</$sc{$_}>" if ! $opened{$_}; $_; } sub reg_fix { my $line=shift; %counts = (); %opened = (); $line =~ s/<([^>]*<)/\&lt;$1/g; # Replace unmatched < with &lt; my @elems = split '(<.*?>)', $line; $line =~ s/<.*?>//g; $counts{$_} = @{[$line =~ m/$_/g]} for keys %sc; for my $elem ( grep !/^</, @elems ) { $elem =~ s#$_#&track#eg for + keys %sc; } join '',@elems; } print reg_fix($_) for <DATA> __DATA__ go/to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> /th +x/ <img src='http://allpoetry.com:8080/images/smile/happy.gif'> _sill +y *to_* Well, if < _$20_, you *need* <a href="...">this</a>

    Which produces :

    go<em>to <img src='http://allpoetry.com:8080/images/smile/happy.gif'> +</em>thx/ <img src='http://allpoetry.com:8080/images/smile/happy.gif' +> <u>silly <b>to</u></b> Well, if &lt; <u>$20</u>, you <b>need</b> <a href="...">this</a>

    (Update - simplified code, egregious abuse of $_)