use strict;
use WWW::Mechanize;
use HTTP::Cookies;

my $mech      =  WWW::Mechanize->new( agent =>"Mozilla/5.0 (X11; U; Li
+nux i686; en-US; rv:1.7) Gecko/20040918 Firefox/0.9.3 ");
my $usr  = "80degreez\@gmail.com";
my $pw  = "mypw";
my $currPage = 1;
my $numPro = 0;
my $loopa = 0;
my $fn = 2;
my $proList = ">>profiles.out";
open (OUT, ">>file.dump" );



$mech->get("http://indiecharts.com/");
$mech->cookie_jar( HTTP::Cookies->new() );

if( $mech->content() =~ /<form Method="Post" action="login.asp"> / ){
        $mech->form_number($fn);
        $mech->field( UserName => $usr );
        $mech->field( Password => $pw );
        $mech->submit();

        if ($mech->success() ) {
            if ( $mech->content() =~ /Pick your IndieCharts Name/ ) {
                    print "User $usr logged in successfully!\n" ;


                        do {

                        $mech->get("http://indiecharts.com/indie_Music
+_Artists.asp?Keyword=&Page=$currPage&butname=");
                               my @profiles = $mech->find_all_links( t
+ext_regex => qr/^d{9}$/ );
                                chomp @profiles;
                                foreach my $parse (@profiles) {
                                        if (@profiles){
                                             $numPro++;
                                             open (OUT, $proList) || p
+rint "cant open profiles file!";
                                             print OUT "$parse\n";
                                             }
                                }
                        print "$currPage $numPro\n\n";
                        $currPage++;
                        } while ( $currPage <= 500 );

        }
        else {
           print " User $usr was unable to log in successfully!\n";
        }
    }
}
else {
  print $mech->content;
  }
  close(OUT);

[download]
no this is not for spam, it's just the way im attempting to familiarize myself with perl pattern matching. Logs in correctly, fetches page correctly, doesn't match/parse links correctly

In reply to Page Scraping by 80degreez

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.