nase has asked for the wisdom of the Perl Monks concerning the following question:

Sorry that this is so long-winded but I am at one of those hallowed early stumbling blocks when learning a new language. I am looking to develop a baseball web site eventually. I was doing great in my research jumping from excel to xml to mysql but then I ran into programming languages and I'm now getting my ass kicked.

Below is the exact script taken out of a Safari book called Baseball Hacks. As opposed to going through the Perl documentation, I am hoping to work off of examples like this one, tweaking them to my needs, and referring to the documentation as I need it. I just can't get this script to work so I am stuck in even beginning that process.

The script below should extract data from the mlb.com site found about halfway down through the code and load it into a mysql database (already created) with the help of other files (such as save_to_db which you will see in the script).

The issue is that I run the script and just get another command line. There is no error. When I run it under ModPerl, I just get a blank white screen. I checked the error log and nothing appears when I execute this. I have run a number of small sample scripts that I have taken from various tutorials with success. This is the first complex one with which I have worked (and note that I did change the user and pass here for safety). And I realize that I went from the wading pool to the deep end with this.

Q1 part I: Whether or not the issue raised in part 2 is the problem, how does one diagnose this without an actual error message? Or maybe a better question is what it means when I execute a script and immediately just get the command line prompt again.

Q1 part II: I'm wondering if I need to pre-set directories. With 'use,' does the call item always have to be in the same folder? Do I need to set a directory for each call item (ex: Mysql, line 10- mysql is in a different directory than the script) before writing that kind of code? I've read a little bit about @INC but haven't delved deeply yet.

Question 2: I realize this is a reach so don't struggle to answer it but any other clues as to how I might get this to run successfully?

#!/usr/bin/perl use strict; use warnings; # loads a database with this season's box scores up through yester +day's games use save_to_db; use Mysql; my $dbh = Mysql->connect('localhost', 'boxes', 'username', 'passwo +rd'); $dbh->{'dbh'}->{'PrintError'} = 1; use XML::Simple; my $xs = new XML::Simple(ForceArray => 1, KeepRoot => 1, KeyAttr = +> 'boxscore'); use LWP; my $browser = LWP::UserAgent->new; # to prevent partial loading of games in progress, only load a day +'s games # after 12:00 PM GMT use Time::Local; my $mintime = timegm(0,0,0,3,3,105); my $maxtime = time() - 60*60*36; my $mintimestr = gmtime(timegm(0,0,0,3,3,105)); my $maxtimestr = gmtime(time() - 60*60*36); print "iterating from $mintimestr to $maxtimestr\n"; for ($i = $mintime; $i <= $maxtime; $i += 86400) { ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = gmtime( +$i); $fmon = length($mon + 1) == 1 ? '0' . ($mon + 1) : ($mon + 1); $fday = length($mday) == 1 ? '0' . $mday : $mday; # build base url for fetching box scores and start fetching $baseurl = 'http://gd2.mlb.com/components/game/mlb/year_' . (19 +00 + $year) . '/month_' . $fmon . '/day_' . $fday . '/'; my $response = $browser->get($baseurl); die "Couldn't get $baseurl: ", $response->status_line, "\n" unless $response->is_success; my $dirhtml = $response->content; while($dirhtml =~ m/<a href=\"(gid_.+)\/\"/g ) { my $game = $1; print "fetching box score for game $game\n"; my $boxurl = $baseurl . $game . "/boxscore.xml"; my $response = $browser->get($boxurl); # die "Couldn't get $boxurl: ", $response->status_line, "\n" unless ($response->is_success) { print "Couldn't get $boxurl: ", $response->status_line, "\n" +; next; } my $box = $xs->XMLin($response->content); save_batting_and_fielding($dbh, $box); save_pitching($dbh, $box); save_game($dbh, $box); my $playersurl = $baseurl . $game . "/players.txt"; my $response = $browser->get($playersurl); unless ($response->is_success) { print "Couldn't get $playersurl: ", $response->status_line, "\ n"; next; } save_roster($dbh, $box, $response->content); } # be a good spider and don't take up too much bandwidth sleep(1); }
I'm on a Windows XP machine. The command line that I use to run the script is perl load_db.pl. When I run it, I know I am in the correct directory where all of the scripts involved are located. I have no idea how it figures out where mysql is but I don't get an error. When I run perl -v, I get a description about Perl 5.8.8. When I run the second message, I get this: _to_db::VERSION Thanks for your help, Adam

Replies are listed 'Best First'.
Re: Blank result but no error
by GrandFather (Saint) on Aug 07, 2007 at 05:11 UTC

    What is the command line you use to "run the script" and under what OS?

    If you run:

    perl -v

    what do you get? What do you get when you run:

    perl -M"save_to_db" -e "print qq/$save_to_db::VERSION/"

    DWIM is Perl's answer to Gödel
Re: Blank result but no error
by GrandFather (Saint) on Aug 07, 2007 at 21:40 UTC

    I don't understand the result you got from the last test. I suspect you mistyped the line. You should have either received a version number or an error message.

    I suspect that there are files that you require that are not present. In particular, have you a file called "save_to_db.pm" in the folder where you are working? Have you installed the MySQL.pm module?


    DWIM is Perl's answer to Gödel
Re: Blank result but no error
by graff (Chancellor) on Aug 08, 2007 at 03:32 UTC
    It looks like the web data that you are shooting at really exists and has the properties expected in the script. But I'm curious about the timing of the script's behavior.

    Given today's date, the settings for $mintime and $maxtime yield a span of about 28 months (from April 3 2005 to Aug. 8 2007), or about 850 days, give or take. In your "for" loop, which iterates over that span one day at a time, there is a "sleep(1)" (very polite).

    Does the script run for nearly 15 minutes and produce no output whatsoever? Or does it run for a very short time?

    Apart from that, the use of a module called "Mysql" seems outdated. You should be using a module called DBI, which in turn uses a "driver" module called DBD::mysql.

    As for the "save_to_db" module, that's not on CPAN, and the only people who have encountered it are probably people who have read that same "Safari book called Baseball Hacks". The problem might be in that module, but you haven't shown it here (and I don't have a copy of that book).

    There is a print statement that precedes the for loop, and if you are not seeing a line like "iterating from Sun Apr 3 00:00:00 2005 to ...", then the script must exiting before it reaches that print statement. (This would also suggest that the script is exiting very quickly, rather than running for 15 minutes.)

    You could try stepping through it with the perl debugger; run it at the command line like this:

    perl -d name_of_script
    You'll be prompted by the debugger for things to do; you can start with "h" to see a brief summary of debugging commands (like "s" to step, "b line#" to set a breakpoint, "c" to continue execution, etc.) You might also find it worthwhile to read the output of "perldoc perldebug". You'll get copious warnings and diagnostics as you step along this way, and you can look up messages you get in the perldiag manual.