dannoura has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I'm trying to search this site and retrieve the results. The problem is that to get all the results I need to press the "Next" button, which has the javascript:

javascript:var frm=document.frmQueryBox; frm.inputpage.value=2;Go('Pager');

Where "2" is the page number.

Is there any way to activate this script with perl?

Replies are listed 'Best First'.
Re: activating javascript with perl
by PodMaster (Abbot) on Jun 30, 2003 at 02:53 UTC
[reply]
Re: activating javascript with perl
by antirice (Priest) on Jun 30, 2003 at 07:35 UTC

    I decided to do some investigative work for you since I have nothing to do right now at 2:00 in the AM. I decided to capture a few submissions from the browser (IE 6.0) and analyze what was submitted.

    Since this is long as all fling flang, I employed a readmore.

    Read more... (5 kB)

    Hope this helps. Pray that they never change the interface :-P

    antirice    
    The first rule of Perl club is - use Perl
    The     ith rule of Perl club is - follow rule i - 1 for i > 1

[reply]
[d/l]
[select]
Re: activating javascript with perl
by Zaxo (Archbishop) on Jun 30, 2003 at 02:50 UTC

    You should do the cpan search JavaScript to see if there is something you can use.

    After Compline,
    Zaxo

[reply]
Re: activating javascript with perl
by zakzebrowski (Curate) on Jun 30, 2003 at 10:59 UTC
    I'm not sure what you're doing, but be careful of legal stuff...
    Seee : their disclaimer page, esp. the Copyright status section...


    ----
    Zak
    Pluralitas non est ponenda sine neccesitate - mysql's philosphy
[reply]
Re: activating javascript with perl
by jmanning2k (Pilgrim) on Jun 30, 2003 at 18:02 UTC
    I didn't think I'd be able to answer this one, but then I saw the site you were trying to crawl.
    NCBI has a nice API for retrieving data - you don't have to walk through the pages.

    The real solution here is to use some modules specifically designed for getting data from NCBI. These modules use their API properly, instead of fudging through javascript. I suggest Bio::Perl for genomic data (I think it can do pubmed articles too -- ah yes, Bio::Biblio - they even have a sample script for PubMed queries included with the bioperl distribution), or NCBI's own Entrez Programming Utilities.

    Personally, I use the Bio::Perl modules on a daily basis for a great deal of the work I do with NCBI data.

    As for legal status, the data is freely available. They do specifically ask you use their API rather than spidering the pages. ;-)
[reply]

      That was useful. Thanks.

      A few minutes ago I came across this module, which is a backend to WWW::Search and seems promising.

      Does anyone have any experience with these modules?

      Any examples would be appreciated.

[reply]

Back to Seekers of Perl Wisdom