in reply to Project Help: Mechanize::Firefox - Scraping Websites with Javascript

The code you've shown is not valid Javascript, so you can't get at it in a convenient way.

If what you've shown actually is a mangled version of what otherwise would be valid Javascript, the easiest way is to pull over the value to Perl space:

my $myContests= $mech->eval_in_page('myContests');

print $myContests->{n};
print $myContests->{id};

[download]
  • Comment on Re: Project Help: Mechanize::Firefox - Scraping Websites with Javascript
  • Download Code

Replies are listed 'Best First'.
Re^2: Project Help: Mechanize::Firefox - Scraping Websites with Javascript
by jdlev (Scribe) on Sep 23, 2014 at 15:08 UTC
    I keep getting the error: Use of uninitialize value when I run this code. So I'm not sure if the variable "packagedContests" is invalid or if it has something to do with how the page is loading?

    use warnings;
use WWW::Mechanize::Firefox;
use DBI;
use Data::Dumper;

    my $url = ("https://www.draftkings.com/contest-lobby");
    my $mech = WWW::Mechanize::Firefox->new();
    $mech->get( $url ) or die("Can't Get Web Page!");

    $contest = $mech->eval_in_page('packagedContests');
  
    print $contest->{n};
    print $contest->{id};

    [download]
    I love it when a program comes together - jdhannibal
[reply]
[d/l]

      I'm sorry - I should read the documentation closer. The code to fetch the variable should be:

      my ($contest, $type) = $mech->eval_in_page( 'packageContests' );

      [download]

      But you should also use strict; at the top of your program. This would have immediately shown the wrong use of ->eval_in_page, as you get a number back instead of a proper reference.

[reply]
[d/l]
[select]
        No Worries. I think it was actually my fault. Let me ask you this. This variable "packagedContests" has literally thousands of child Objects associated with it. Is there someway to store the variable as a hex, and then iterate through it to break it apart? Also...FWIW...this is the DOM object that I found that was associated with the variable using firebug. It lists the following elements within each Object. The bigger problem is this is just one collection of about 2k of these objects I'll need to eventually parse through. Using my console, I'm able to select the specific elements of each object with the following code: packageContests[0].n Any Thoughts?

        a    27    
attr    Object { IsGuaranteed="true", IsStarred="true"}    
ec    0    
fpp    108    
fwt    false    
id    1141705    
isOwner    false    
m    92400    
mec    999999
n    "NFL $2.2M Millionaire Maker [$1,000,000 to 1st!]"    
nt    3481    
po    2200000    
pt    1    
rl    false    
rlc    0    
rll    9999    
s    1    
sa    false
sd    "/Date(1412528400000)/"
sdstring    "Sun 1:00PM"    
so    -99999999
startTimeType    0    
tix    false
tmpl    28934    
uc    0    
ulc    0

        [download]
        I love it when a program comes together - jdhannibal
[reply]
[d/l]

In Section Seekers of Perl Wisdom