Javascript is not the problem it is access to the DOM. For simple javascript you can do something like this:

our @URLS;
my $runtime = JavaScript::Runtime->new();;
my $context = $runtime->create_context();
$context->bind_function(
  name => 'open', func => sub {
    push(@URLS, @_);
    return 0;
} );
[download]

Then

$context->eval(q(open("http://bob.com/")))
[download]

will push http://bob.com/ onto @URLS.
-- gam3
A picture is worth a thousand words, but takes 200K.

Expanding on Joost's comments, I have so far worked on page by page basis using Live HTTP headers in firefox to work out what is being sent by the browser after the java script processing.
Someday I'd like to try JavaScript::SpiderMonkey. shmem discussed it here.

How can a proxy be set with those clones?

AFAIK those clones honour the http_proxy environment variable, e.g. on UNIX (sh, bash,...):

$ export http_proxy=http://proxy.example.com:3128/
$ perl myclonescript.pl
[download]

then the "clone" in the myclonescript.pl will connect any site through proxy.example.com.

Or set it inside the script:

BEGIN {
   $ENV{http_proxy} = 'http://proxy.example.com:3128/';
}
use WWW::Mechanize
...
[download]

Note that the environment variable must be set before any "clone" module is loaded.

--shmem

_($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Mechanize uses LWP::UserAgent, so you can set proxies the same way.

---




($_='kkvvttuu bbooppuuiiffss qqffssmm iibbddllffss')
=~y~b-v~a-z~s; print

but what about mechs ? How can you get that link ?

You haven't given me much to work on here but in any event let me point out WWW::Mechanize, HTML::TokeParser and friends. If you have an idea of what patterns of response you are getting it should not be that big a deal to slurp out the built up link from the response string and then follow it.