in reply to Web Site Mapper

You may want to change:

sub DISALLOWED_SCHEMES ()  { qw( mailto ) }

to:

sub DISALLOWED_SCHEMES ()  {
  qw( mailto javascript )
}

[download]

Otherwise your script will try to spider javascript functions. This means you're also going to need to change:

next if grep { $uri->scheme eq $_ } DISALLOWED_SCHEMES;

[download]

to something like:

my @dissallowed = DISALLOWED_SCHEMES;
foreach my $scheme (@dissallowed) {
  next MAINLOOP if ($uri->scheme eq $scheme);
}

[download]

I forget whether or not a next works in a for loop, and don't really have time to look it up, so I used MAINLOOP and assumed the loop we're nexting is named.

Want to support the EFF and FSF by buying cool stuff? Click here.

Edited by BazB: close stray code tag.

Replies are listed 'Best First'.
Re: Re: Web Site Mapper
by hardburn (Abbot) on Feb 16, 2004 at 16:09 UTC

    I could see how adding javascript to the disallowed schemes would be a good idea, but I don't see what the foreach addition would provide over the current grep solution. Seems like a lot more code to do the exact same thing.

    BTW--In the foreach case, you're right about needing an explicit label.

    ----
    : () { :|:& };:

    Note: All code is untested, unless otherwise stated

[reply]
[d/l]
[select]
      Ooops, you're right. I was looking over the code quickly on my way to work and saw $_ and added in my 2 cents.
      Want to support the EFF and FSF by buying cool stuff? Click here.
[reply]

In Section Code Catacombs