Re^3: POST fieldnames with underscores

Good, it works for you, but I am not convinced, your program works as intended. Consider:

Token in HTTP-header (your current fix):

my $res = $ua->request(POST $url,
                       'user-agent'                  => 'Mozilla/5.0',
                       ':member_verbosity_threshold' => 0,
                       ':csrf_token'                 => '2802000000...
+',
#-- not required:      'content-type'                => 'application/x
+-www-form-urlencoded',
                      );
[download]

Result:

POST /xxx HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: localhost:5000
User-Agent: Mozilla/5.0
Content-Length: 0
Content-Type: application/x-www-form-urlencoded
csrf_token: 28020000006.....
member_verbosity_threshold: 0
[download]

Token in HTTP-body (content) - this is more like what your <FORM ...> example would do:

 my $res2 = $ua->request(POST $url,
                       'user-agent'                  => 'Mozilla/5.0',
                       ':member_verbosity_threshold' => 0,
#-- not in header:     ':csrf_token'                 => '28020000006..
+...',
                       Content                       => [ csrf_token =
+> '28020000006.....', ]  #-- but in body
                      );
[download]

Result:

POST /xxx HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: localhost:5000
User-Agent: Mozilla/5.0
Content-Length: 27
Content-Type: application/x-www-form-urlencoded
member_verbosity_threshold: 0

csrf_token=28020000006.....
[download]

Hope, that shed some light. I was also going to nitpick, that you should prepend your non-standard headers with 'X-' (like X-Csrf-Token), but that practise seems discouraged since a couple of years: SO: Custom HTTP headers : naming conventions. Thanks, I learned something new ;-)

Comment on Re^3: POST fieldnames with underscores Select or Download Code

Replies are listed 'Best First'.
Re^4: POST fieldnames with underscores by BernieC (Pilgrim) on Aug 12, 2018 at 18:54 UTC
OK, I'm real confused now. I notice that my form has, in fact, stopped working. to clarify: I only noticed that there were underscores -- I didn't notice that the colons were still there and that, in fact, the program didn't work}. My guess is that the server doesn't like the colons.. :o) {As for the X-, not my choice: I simply scrape the form I get and try to return the right fields}. I didn't understand what you told me about the content/headers stuff. Here is {part of the} actual form I fetched <FORM action="...members/add" method="POST" enctype="multipart/form-da +ta"> <input type="hidden" name="csrf_token" value="280200000069c97e705b7347 +000000746573746c6973742d66616e746173796661726d2e636f6d2b61646d696e3a3 +763663438346436393036323038363738613566383231623461323132353762356361 +3763633665"> [...] <INPUT name="send_notifications_to_list_owner" type="RADIO" value="0" +> <INPUT name="send_notifications_to_list_owner" type="RADIO" value="1" + CHECKED > <TEXTAREA NAME=subscribees ROWS=10 COLS=70%></TEXTAREA> [...] <INPUT name="setmemberopts_btn" type="SUBMIT" value="Submit Your Chang +es" > [download] Then what I do is set up a @fields array with `@fields= ( ":csrf_token" => $csrf_token subscribees => "$name <$email>", ":setmemberopts_btn" => 'Submit Your Changes', ":send_notifications_to_list_owner" => "1", ); $res = $ua->request(POST $_[0], \@fields);` [download] and that not only doesn't work, but gets me an ugly `<h3><strong><font color="red" size="+2">Error: </font></strong><em>The + form lifetime has expired. (request forgery check)</em></h3><center> +<h2>Testlist mailing list administration<br>Membership Managemen +t... Section</h2></center><hr>` [download] I'm not sure what they check for the 'lifetime' but I bet it is the csrf_token. and I see in the request that every field name now has a colon in front of it {except for the fields that showed up in the request-content; they all have %3's in front of them}. So I feel trapped; is there a way to pass the underscores through to the server, but without the colons?? {short of my going into HTTP::Headers and actually tweaking the code to never "canonicalize"} 2018-08-13 Athanasius fixed code tag placement	[reply] [d/l] [select]
Re^5: POST fieldnames with underscores by Perlbotics (Archbishop) on Aug 12, 2018 at 20:15 UTC
I didn't understand what you told me about the content/headers stuff. That's why I pointed out the distinction between data conveyed in the header and data conveyed in the content/body. IF you want to mimic the form submission, you must not transfer the data in the header - no browser does that! Perhaps, the csrf_token might need to be transferred with the header but that is surely defined in the fine API documentation, right? I GUESS, you need something like this: #-- everything from the FORM here (nerver use colons here): my@form_fields= ( csrf_token => $csrf_token, #-- above looks right, according to form example subscribees => "$name <$email +>", setmemberopts_btn => 'Submit Your C +hanges', send_notifications_to_list_owner" => "1" ); #-- read the API-doc: IF the csrf_token is required in the header # (use colons to prevent canonicalisation): my @headers = ( ':csrf_token' => $csrf_token, ':member_verbosity_threshold' => 0, 'User-Agent' => 'Mozilla/5.0' ); my $res = $ua->request(POST $_[0], @headers, Content => \@form_fields ); [download] Regarding the lifetime expiration: Of course, you must provide the currently valid csrf_token. I hope, running this application is in accordance with laws and terms of use...	[reply] [d/l]
Re^6: POST fieldnames with underscores by marto (Cardinal) on Aug 13, 2018 at 07:08 UTC
"I hope, running this application is in accordance with laws and terms of use..." My guess based on the data so far is that they're trying to automate gnu mailman via it's web interface as a standard user, rather than with one of the other interfaces.	[reply]