bigswifty00000 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks!

I can't seem to get to the shopping list page of OOM. 'http://outofmilk.com/ShoppingList.aspx?signin=1'

I've tried a 1000 variations (witness all the sloppy commented code). I can't seem to get past the login page.

When I use the chrome debugger it looks like the POST is executing a GET to get to the shopping list page.

Could not figure out how to do that.

Sorry about the lengthy code, but I wanted to show some of what I tried.</p

I have very little experience with web apps. Please help! Thanks!

#!/usr/bin/env perl use strict; use warnings; use Data::Dumper; use LWP; use WWW::Mechanize; use Data::Printer colored => 1; my $username = 'email'; my $password = 'password'; my $url = 'https://outofmilk.com'; my $list_url = 'http://outofmilk.com/ShoppingList.aspx?signin=1'; my $mech = WWW::Mechanize->new( autocheck => 0, timeout => 30, ); $mech->agent( "Mozilla/5.0 (Windows NT 6.1)" ); $mech->credentials( $username, $password ); $mech->get($url); print Dumper( $mech->title ); p $mech->response->status_line; my $current = $mech->current_form(); #print Dumper($current); $mech->set_visible( $username, $password ); p $mech->response->status_line; $mech->submit(); p $mech->response->status_line; #print Dumper( $mech->title ); #exit; $mech->get($list_url); p $mech->response->status_line; print Dumper( $mech->title ); #exit; =pod my $result = $mech->submit_form( form_number => 1, fields => { 'id_login-email' => $username, 'id_login-password' => $password, }, ); print $result->content(); =cut # exit; $mech->field( 'login-email' => $username ); $mech->field( 'login-password' => $password ); $mech->submit(); p $mech->response->status_line; exit; p $mech->title; p $mech->res->request->as_string; exit; p $mech->content(); p $mech->res->request->as_string;

Replies are listed 'Best First'.
Re: Out of milk website via WWW::Mechanize
by LanX (Saint) on Dec 30, 2016 at 16:46 UTC
    Quick guess: Did you make sure the site can be used without JavaScript being enabled?

    Because WWW::Mechanize doesn't do JS. (see FAQ )

    On a side note: does the website allow automatic retrieval?

    Otherwise IMHO it wouldn't be legal to help you.

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)
    Je suis Charlie!

      Thanks for the reply, I was aware that Mech doesn't do js, but thought if not trying to manipulate js, I could get through to the shopping list page.

      As far as legal concern, this was just to manipulate my own personal lists on the site, (which I can do manually) and wouldn't be used as a bot of any kind. It was more of a learning experience.

        Sorry, but did you answer my question?

        Does the website work within a browser with deactivated JS? Yes or no?

        If not, forget WWW::Mechanize !

        > but thought if not trying to manipulate js, I could get through to the shopping list page.

        ? ? ?

        Mech doesn't do JS at all!

        Cheers Rolf
        (addicted to the Perl Programming Language and ☆☆☆☆ :)
        Je suis Charlie!

Re: Out of milk website via WWW::Mechanize
by thomas895 (Deacon) on Dec 30, 2016 at 20:47 UTC

    You're trying to send your credentials as HTTP authentication, which the website probably doesn't support. I haven't looked, but it probably uses aweb form to log in. Use your browser's inspector to observe what happens when you log in, and then automate that.

    -Thomas
    "Excuse me for butting in, but I'm interrupt-driven..."

      Thanks for the reply.

      Yea, that was just one of the things I attempted. It is a web form, and I used a post, but get a 404 error when it posts to a url https://outofmilk.com/action which doesn't exist, and doesnt make it to 'http://outofmilk.com/ShoppingList.aspx?signin=1' which is where I'm trying to get to.