For the curious, I'll post what I ended up doing. One of the
main requirements of this script was that it carefully preserved
the formatting of the files that it ran on- the only change could
be the addition of the url markers, so that influenced
my solution. the script would be called with two files- one html
and one plain text.
Sample html file to call with:
this a link: <br> <a href="http://www.somewhere.org/foo">somewhere</a>
+<p> <a href= https://somewhere.else.net/bar/>another place to go</a>.
+<p>
And sample text file pair to the html file:
this a link:
http://www.somewhere.org/foo
another place to go:
https://somewhere.else.net/bar/
so here's my code to tag the urls in these files:
#$unprocessed_html holds the text of the html version of my file
#$unprocessed_text holds the plain text version of my file
#@markers holds the unique url markers, ie [/qfk33pe][/nnd92093]
#handle html version
my $processed_html = "";
while (length($unprocessed_html) > 0) {
if($unprocessed_html =~ s/^(.*?\b(href|action)\s*=\s*)//si) {
$processed_html .= $1;
} else {
$processed_html .= $unprocessed_html;
last;
}
if (not(@markers)){
$processed_html .= $unprocessed_html;
warn "no more markers available for remaining links\n";
last;
}
if($unprocessed_html =~ s/^([^\"\'][^<>\s]*)//) {
my $url = $1 . shift(@markers);
#strip double //s due to sloppy input
$url =~ s|//|/|g;
$processed_html .= $url;
} elsif($unprocessed_html =~ s/^([\"\'])([^<>]*?)\1//) {
my $url = $1 . $2 . shift(@markers) . $1;
#strip double //s that can result from sloppy input
$url =~ s|//|/|g;
$processed_html .= $url;
} else {
die "something happened here";
}
}
#handle text version
my $processed_text = "";
while (length($unprocessed_text) > 0) {
if($unprocessed_text =~ s/^(.*?\b)http/http/si) {
$processed_text .= $1;
} else {
$processed_text .= $unprocessed_text;
last;
}
if (not(@markers)){
$processed_text .= $unprocessed_text;
warn "no more markers available for remaining links\n";
last;
}
if($unprocessed_text =~ s/(http[\S]+)\s+//){
my $urlfound = $1 . shift (@markers);
$urlfound =~ s|//|/|g;
$processed_text .= "$urlfound\n";
} elsif($unprocessed_text =~ s/(http[\S]+)$//){
my $urlfound = $1 . shift (@markers);
$urlfound =~ s|//|/|g;
$processed_text .= "$urlfound\n";
}
}
-- cat