Something like this:

use warnings;
use strict;

use URI::Find;

my $data = join '', <DATA>;

$data =~ s/(?<=[.:\/?&])\n//g;
$data =~ s/\n(?=[.:\/?&])//g;

my $finder = URI::Find->new(
                 sub { print "Found: $_[0]\n"; });

$finder->find(\$data);

__DATA__
There's a URL http://here.com and a URL http://
there.com. What else? A really long one http://abc
.def.com?foo=bar&foo=bar&foo=bar&foo=bar&foo=bar&foo=bar&
bar=foo. And this
is a regular line break.
[download]

Does anyone here have any idea how to construct a regular expression to match an URL broken in this way, with an linebreak at an arbitrary position?

(?:(?:http)://(?:(?:(?:(?:(?:(?:[a-zA-Z0-9][-a-zA-Z0-9]*)?[a-zA-Z0-9])
[.])*(?:[a-zA-Z][-a-zA-Z0-9]*[a-zA-Z0-9]|[a-zA-Z])[.]?)|(?:[0-9]+[.][0
-9]+[.][0-9]+[.][0-9]+)))(?::(?:(?:[0-9]*)))?(?:/(?:(?:(?:(?:(?:(?:[a-
zA-Z0-9\-_.!~*'():@&=+$,]+|(?:%[a-fA-F0-9][a-fA-F0-9]))*)(?:;(?:(?:[a-
zA-Z0-9\-_.!~*'():@&=+$,]+|(?:%[a-fA-F0-9][a-fA-F0-9]))*))*)(?:/(?:(?:
(?:[a-zA-Z0-9\-_.!~*'():@&=+$,]+|(?:%[a-fA-F0-9][a-fA-F0-9]))*)(?:;(?:
(?:[a-zA-Z0-9\-_.!~*'():@&=+$,]+|(?:%[a-fA-F0-9][a-fA-F0-9]))*))*))*))
(?:[?](?:(?:(?:[;/?:@&=+$,a-zA-Z0-9\-_.!~*'()]+|(?:%[a-fA-F0-9][a-fA-F
0-9]))*)))?))?)
[download]

Abigail

/http:\S*[^\w\s]/g and s/\G\n//;
[download]

/http:\S*[^\w\s]/g and s/\G\n// and $http=1;
$http and /^\S*[\w\s]/g and s/\G\n// or $http=0;
[download]

Check out
http://homestarrunner.com/sbemail/
and let me know what you think
[download]

Update: Oops, forgot the /g modifier on the pattern matches, so the \G didn't work.

[^\w\s] matches all characters that are not both a word character and a whitespace character. That is, it matches *any* character. Which means that from the string <URL:http://www.example.com/> your first regex is going to match http://www.example.com/>. Your second regex is only going to match if the string contains http: followed by a sequence of non-whitespace characters, followed by a whitespace character, followed by a newline. Given the string http://www.example.com/\npath/here, your solution will not set $http to 1.

Abigail

Astonishingly, you are wrong about the character class. [^\w\s] matches characters that are neither words nor whitespace; i.e., punctuation:

$_='o.ne<a>tw/o<b>';
@punks = /[^\w\s]/g;
print "<@punks>";
[download]

yields

<. < > / < >>
[download]

The problem with my example was that I left the /g modifier off the pattern match. I've updated it, and tested it:

while(<DATA>){
/http:\S*[^\w\s]/g and s/\G\n//;
print;
}
__DATA__
there is an http://whatever.com/address/
crossing/line/boundaries.html right in the
middle of this nice string.
[download]

---

The PerlMonk tr/// Advocate

For example, none of the suggested regex's allow for a secure server URL ... https

Just the tip of the iceberg, it seems ;)

Hagbone

Thanks everyone -- including the sceptics -- for your help with this one.

I think the problem may not be amenable to any real solution. Although we would be happy with a solution that matched most broken URIs most of the time (or even some of the time), we can't accept the risk of creating new broken URLs -- and unfortunately, that could easily happen.

Take this simple example:

Here is a valid link http://mydomain.org/
which is not broken

I can't see any way of detecting whether http://mydomain.org/ is correct or whether it should really be http://mydomain.org/which

It would work if we could be certain that the first token on the next line -- the one to be appended to the possibly-broken URL -- could only exist as an end-fragment of a syntactically-correct URL. I suspect that even if theoretically possible, it would be impractical.