in reply to Recommendations on a module for web crawling

I've tried using Mechanize::Firefox but the page complains about unsupported browser.

Assuming you mean WWW::Mechanize::Firefox, then yes that would be the first port of call. If the site kicks you out perhaps they don't want any bots at all - check their Ts and Cs and honour them.

Without any specifics there's little more to suggest.

  • Comment on Re: Recommendations on a module for web crawling

Replies are listed 'Best First'.
Re^2: Recommendations on a module for web crawling
by edimusrex (Monk) on Jul 25, 2016 at 18:36 UTC

    Thanks for the response. I am just trying to monitor a site for when it updates its inventory. The javascript must be building the list from a database before displaying it. I will look into WWW::Mechanize::Firefox some more and see if I can't dig a little deeper.

[reply]

In Section Seekers of Perl Wisdom