in reply to Re: Project Help: Mechanize::Firefox - Scraping Websites with Javascript
in thread Project Help: Mechanize::Firefox - Scraping Websites with Javascript

I keep getting the error: Use of uninitialize value when I run this code. So I'm not sure if the variable "packagedContests" is invalid or if it has something to do with how the page is loading?

use warnings;
use WWW::Mechanize::Firefox;
use DBI;
use Data::Dumper;

    my $url = ("https://www.draftkings.com/contest-lobby");
    my $mech = WWW::Mechanize::Firefox->new();
    $mech->get( $url ) or die("Can't Get Web Page!");

    $contest = $mech->eval_in_page('packagedContests');
  
    print $contest->{n};
    print $contest->{id};

[download]
I love it when a program comes together - jdhannibal
  • Comment on Re^2: Project Help: Mechanize::Firefox - Scraping Websites with Javascript
  • Download Code

Replies are listed 'Best First'.
Re^3: Project Help: Mechanize::Firefox - Scraping Websites with Javascript
by Corion (Patriarch) on Sep 23, 2014 at 15:28 UTC

    I'm sorry - I should read the documentation closer. The code to fetch the variable should be:

    my ($contest, $type) = $mech->eval_in_page( 'packageContests' );

    [download]

    But you should also use strict; at the top of your program. This would have immediately shown the wrong use of ->eval_in_page, as you get a number back instead of a proper reference.

[reply]
[d/l]
[select]
      No Worries. I think it was actually my fault. Let me ask you this. This variable "packagedContests" has literally thousands of child Objects associated with it. Is there someway to store the variable as a hex, and then iterate through it to break it apart? Also...FWIW...this is the DOM object that I found that was associated with the variable using firebug. It lists the following elements within each Object. The bigger problem is this is just one collection of about 2k of these objects I'll need to eventually parse through. Using my console, I'm able to select the specific elements of each object with the following code: packageContests[0].n Any Thoughts?

      a    27    
attr    Object { IsGuaranteed="true", IsStarred="true"}    
ec    0    
fpp    108    
fwt    false    
id    1141705    
isOwner    false    
m    92400    
mec    999999
n    "NFL $2.2M Millionaire Maker [$1,000,000 to 1st!]"    
nt    3481    
po    2200000    
pt    1    
rl    false    
rlc    0    
rll    9999    
s    1    
sa    false
sd    "/Date(1412528400000)/"
sdstring    "Sun 1:00PM"    
so    -99999999
startTimeType    0    
tix    false
tmpl    28934    
uc    0    
ulc    0

      [download]
      I love it when a program comes together - jdhannibal
[reply]
[d/l]

        What do you mean by "a hex"?

        You will need to learn Javascript and understand how Javscript types map to Perl.

        If you mean to access packageContests[0].n instead of packageContests.n, that means you need to do it different on the Perl side as well. You will need to treat packageContests as array reference instead of a hash reference:

        print $packageContests->[0]->{n};

        [download]

        The data structures are mapped between Perl and Javascript, but you will need to understand the data structure itself to make use of it.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom