Winning an Intranet competition with the aid of WWW::Mechanize

Our corporate intranet boys at my client site have been busy making over the intranet site, and relaunched it on Wednesday. They also announced a competition with 5 star images at the top of certain pages. They offered a prize for a randomly chosen correct entry.

Of course, the temptation was too much for me to resist :). I learned all about how to do authentication with NTLM from LWP::UserAgent in the process. Here goes:(password suppressed to protect the innocent)

#!perl

use strict;
use warnings;

use WWW::Mechanize;
use HTML::TokeParser;

my @todo = 'http://intranet';
my %seen = (@todo, 1);
my $mech = WWW::Mechanize->new( keep_alive => 1 );
$mech->credentials('intranet:80','','CORP\\williami','censored');

while (my $url = shift @todo) {
    print "Visiting: $url\n";

    $mech->get( $url);

    for ($mech->links) {
        my $link = $_->url_abs;
        push @todo, $link unless exists $seen{$link};
         $seen{$link}++;
    }

    my $text = $mech->content;

    my $tp = HTML::TokeParser->new(\$text);

    while (my $tag = $tp->get_tag('img')) {
        print "Found image: ", $tag->[1]{src}, "\n";
    }
}
[download]

I came back from lunch to find that I had won. I'm curious about how many other correct entries there were though.

--
I'm Not Just Another Perl Hacker

Comment on Winning an Intranet competition with the aid of WWW::Mechanize Download Code

Replies are listed 'Best First'.
Re: Winning an Intranet competition with the aid of WWW::Mechanize by merlyn (Sage) on May 13, 2005 at 16:10 UTC
Why did you use: `my $text = $mech->content; my $tp = HTML::TokeParser->new(\$text); while (my $tag = $tp->get_tag('img')) { print "Found image: ", $tag->[1]{src}, "\n"; }` [download] and not simply: `for ($mech->images) { print "Found image: ", $_->url_abs, "\n" }` [download] See, I would have won. Less typing. :) -- Randal L. Schwartz, Perl hacker Be sure to read my standard disclaimer if this is a reply.	[reply] [d/l] [select]
Re: Winning an Intranet competition with the aid of WWW::Mechanize by mrborisguy (Hermit) on May 13, 2005 at 15:47 UTC
CHEATER! just kidding... good work! at my current job, i probably would have done the same thing. ++	[reply]
Re: Winning an Intranet competition with the aid of WWW::Mechanize by muba (Priest) on May 17, 2005 at 22:57 UTC
Maybe it's a special feature or whatsoever, but... `@array = 'scalar string'; #??? # original: my @todo = 'http://intranet';` [download] What...? `"2b"\|\|!"2b";$$_="the question"` Besides that, my code is untested unless stated otherwise. One more: please review the article about regular expressions (do's and don'ts) I'm working on.	[reply] [d/l] [select]
Re^2: Winning an Intranet competition with the aid of WWW::Mechanize by TheStudent (Scribe) on May 18, 2005 at 00:13 UTC
Because further down in the OP, inside the while @todo is reused to recurse into the links on the page... `while (my $url = shift @todo) { print "Visiting: $url\n"; $mech->get( $url); for ($mech->links) { my $link = $_->url_abs; push @todo, $link unless exists $seen{$link}; $seen{$link}++; } ... }` [download] Lou	[reply] [d/l]
Re^3: Winning an Intranet competition with the aid of WWW::Mechanize by merzy (Scribe) on May 19, 2005 at 01:43 UTC
I think muba was confused by the scalar appearing where he would expect to see an array. So was I, until I went off and tested it. Works like a charm, but I doubt I'll ever use it. (Because it causes reader confusion...) You learn something new every day here in the monastery. :-)	[reply]