tel2 has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I'm using Perl's LWP::UserAgent module and I'd like to check/uncheck checkboxes in GNU Mailman's "Membership List" webpage.  Here's the element that I'm trying to change:

    <td><center><INPUT name="test2%40mydomain.com_notmetoo" type="CHEC
+KBOX" value="off" ></center></td>

[download]
And here's the code I've written in an attempt to check the above "notmetoo" checkbox for the test2@mydomain.com email address:
    #!/usr/bin/perl

    use LWP::UserAgent;

    $ua = new LWP::UserAgent;
    $ua->cookie_jar({ file => "$ENV{HOME}/.mailmanrc" });

    $res = $ua->post('http://mydomain.com/mailman/admin/test1_mydomain
+.com/members/list',
        Content_Type => 'form-data',
        Content => [
            'test2%40mydomain.com_notmetoo' => 'on',
            setmemberopts_btn => 'Submit Your Changes'
        ]
    );
    if ($res->is_success) {
        print $res->decoded_content;
        print "Changed user setting...I hope!\n";
    }
    else {
        die $res->status_line;
    }

[download]
Unfortunately, the above does not work. It prints my "Changed user setting...I hope!" line, but the setting doesn't get changed.

I'm not sure why the "@" is represented by "%40" in the element name (any ideas?), but I've also tried:
    'test2\%40mydomain.com_notmetoo' => 'on',
and
    'test2@mydomain.com_notmetoo' => 'on',
without success.

I've put the HTML which was generated by the "print $res->decoded_content" command here for you.
That output it what I would expect if I had just submitted the form without checking any box.  That is also what the webpage looks like before attempting to check the checkbox, so if you want to see that, it's the same.

Note that I have manually checked the "notmetoo" checkbox for Test1 via my browser, but Test2 is the one I'm trying to check with Perl.

I was able to use the "Find member" search on the same webpage, and that worked fine.  Here's an extract of that (working) code: 

    $res = $ua->post('http://mydomain.com/mailman/admin/test1_mydomain
+.com/members/list',
        Content_Type => 'form-data',
        Content => [
                findmember => 'Test2',
                findmember_btn => 'Search...'
        ]
    );

[download]
I have also been able to sucessfully check other checkboxes on other Mailman webpages.

Questions:
1. Can LWP::UserAgent be used to check the "nottome" checkbox (for example)?
2. What am I doing wrong?

I've spent many hours on this, and I'm getting nowhere fast, so your help would be appreciated.

Thanks.
Tel2

Replies are listed 'Best First'.
Re: Unable to check checkbox via LWP::UserAgent
by Gangabass (Vicar) on Aug 15, 2012 at 00:42 UTC

    May be there is some Javascript code executed on submit? Does this checkbox work in the browser with Javascript disabled?

    Anyway I highly recommend you to try WWW::Mechanize for web automation.

[reply]
[d/l]
      Thanks Gangabass,

      I've now confirmed that checking the boxes and submitting with JavaScript disabled in my browser (Firefox) saves my checkbox selections without a problem, so it seems that wasn't the issue.  I doubt JavaScript is used at all in Mailman's webpages, as it seems to work like straight HTML.

      Why do you recommend WWW::Mechanize?

      Thanks.

[reply]
Re: Unable to check checkbox via LWP::UserAgent
by tobyink (Canon) on Aug 15, 2012 at 06:10 UTC 
    'test2@mydomain.com_notmetoo' => 'off'

    [download]
    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
[reply]
[d/l]
      Thanks tobyink, but did you see the comment in my original post which said:
      "I've also tried:
          'test2\%40mydomain.com_notmetoo' => 'on',
      and
          'test2@mydomain.com_notmetoo' => 'on',
      without success."
      ?

      Your suggestion looks like the last one of mine above, but with 'off' instead of 'on'.  Are you suggesting that setting it to 'off' will check the checkbox?  I thought it would uncheck it.  Anyway, I've now tried your suggestion, and like my previous attempts, it also does nothing.

      I'm guessing that this problem may be caused by the '%' in the element name, but changing the '%40' (hex for '@') to '@' doesn't seem to help.

[reply]
[d/l]
[select]

        "Your suggestion looks like the last one of mine above, but with 'off' instead of 'on'. Are you suggesting that setting it to 'off' will check the checkbox?"

        Precisely.

        Given the following form:

        <form action="foo.cgi" method="post">
  <input name="foo" value="off" type="checkbox">
</form>

        [download]

        Then when the checkbox is ticked, the string foo=off will be posted to the server. When the checkbox is blank, then the empty string is posted to the server. The string "off" doesn't have any significance to the HTML form. If doesn't mean false, or unticked, or anything. It's just a three-character string that will be posted to the server as the value for the "foo" input when that input has been ticked.

        "I'm guessing that this problem may be caused by the '%' in the element name"

        That may well be the cause. Normally when a form input name contained an "@" I'd expect the HTML itself to contain a plain "@" (or perhaps HTML-escaped as &#x40; or &#64;). The escaping as '%40' would happen when the form was submitted and encoded as application/x-www-form-urlencoded. There might be some double-encoding happening at some stage.

        I suggest taking a look with Wireshark or similar to compare what data LWP::UserAgent is posting compared to what a desktop browser posts.

        perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
[reply]
[d/l]
[select]
Re: Unable to check checkbox via LWP::UserAgent
by Anonymous Monk on Aug 15, 2012 at 02:21 UTC

    I've spent many hours on this, and I'm getting nowhere fast, so your help would be appreciated.

    CPAN CPAN CPAN WWW::Mailman

[reply]
      Thanks Anon', but I started with WWW::Mailman, and had the same problem there.  I emailed BooK (the module programmer) but received no response.
[reply]

Back to Seekers of Perl Wisdom