in reply to Re: POST fieldnames with underscores
in thread POST fieldnames with underscores

No wonder I couldn't find it...:o). For example
<FORM action="../../../admin.cgi/testlist-fantasyfarm.com/privacy/send +er" method="POST" > <input type="hidden" name="csrf_token" value="280200000069096e6f5b7347 +000000746573746c6973742d66616e746173796661726d2e636f6d2b61646d696e3a6 +638333163393562386564343865383939386464623764346337663661356161333937 +6135316132">
and so I modified my program to stick a ':" in front of every field and Just as you said - -I tried it and it program *worked*.
'user-agent' => 'Mozilla/5.0', ':member_verbosity_threshold' => 0, ':csrf_token' => '28020000006916706f5b7347000000746573746c6973742d6661 +6e746173796661726d2e636f6d2b61646d696e3a63356234316237373565643432393 +036383931306131616635643239653666366463643239373065', 'content-type' => 'application/x-www-form-urlencoded',
Thanks! I'd have never found that info {since I had no idea what I should be looking for}.

Replies are listed 'Best First'.
Re^3: POST fieldnames with underscores
by Perlbotics (Archbishop) on Aug 12, 2018 at 09:50 UTC

    Good, it works for you, but I am not convinced, your program works as intended. Consider:

    Token in HTTP-header (your current fix):

    my $res = $ua->request(POST $url, 'user-agent' => 'Mozilla/5.0', ':member_verbosity_threshold' => 0, ':csrf_token' => '2802000000... +', #-- not required: 'content-type' => 'application/x +-www-form-urlencoded', );

    Result:

    POST /xxx HTTP/1.1 TE: deflate,gzip;q=0.3 Connection: TE, close Host: localhost:5000 User-Agent: Mozilla/5.0 Content-Length: 0 Content-Type: application/x-www-form-urlencoded csrf_token: 28020000006..... member_verbosity_threshold: 0

    Token in HTTP-body (content) - this is more like what your <FORM ...> example would do:

    my $res2 = $ua->request(POST $url, 'user-agent' => 'Mozilla/5.0', ':member_verbosity_threshold' => 0, #-- not in header: ':csrf_token' => '28020000006.. +...', Content => [ csrf_token = +> '28020000006.....', ] #-- but in body );

    Result:

    POST /xxx HTTP/1.1 TE: deflate,gzip;q=0.3 Connection: TE, close Host: localhost:5000 User-Agent: Mozilla/5.0 Content-Length: 27 Content-Type: application/x-www-form-urlencoded member_verbosity_threshold: 0 csrf_token=28020000006.....

    Hope, that shed some light. I was also going to nitpick, that you should prepend your non-standard headers with 'X-' (like X-Csrf-Token), but that practise seems discouraged since a couple of years: SO: Custom HTTP headers : naming conventions. Thanks, I learned something new ;-)

      OK, I'm real confused now. I notice that my form has, in fact, stopped working. to clarify: I only noticed that there were underscores -- I didn't notice that the colons were still there and that, in fact, the program didn't work}. My guess is that the server doesn't like the colons.. :o) {As for the X-, not my choice: I simply scrape the form I get and try to return the right fields}. I didn't understand what you told me about the content/headers stuff. Here is {part of the} actual form I fetched
      <FORM action="...members/add" method="POST" enctype="multipart/form-da +ta"> <input type="hidden" name="csrf_token" value="280200000069c97e705b7347 +000000746573746c6973742d66616e746173796661726d2e636f6d2b61646d696e3a3 +763663438346436393036323038363738613566383231623461323132353762356361 +3763633665"> [...] <INPUT name="send_notifications_to_list_owner" type="RADIO" value="0" +> <INPUT name="send_notifications_to_list_owner" type="RADIO" value="1" + CHECKED > <TEXTAREA NAME=subscribees ROWS=10 COLS=70%></TEXTAREA> [...] <INPUT name="setmemberopts_btn" type="SUBMIT" value="Submit Your Chang +es" >
      Then what I do is set up a @fields array with
      @fields= ( ":csrf_token" => $csrf_token subscribees => "$name <$email>", ":setmemberopts_btn" => 'Submit Your Changes', ":send_notifications_to_list_owner" => "1", ); $res = $ua->request(POST $_[0], \@fields);
      and that not only doesn't work, but gets me an ugly
      <h3><strong><font color="red" size="+2">Error: </font></strong><em>The + form lifetime has expired. (request forgery check)</em></h3><center> +<h2>Testlist mailing list administration<br>Membership&nbsp;Managemen +t... Section</h2></center><hr>
      I'm not sure what they check for the 'lifetime' but I bet it is the csrf_token. and I see in the request that every field name now has a colon in front of it {except for the fields that showed up in the request-content; they all have %3's in front of them}. So I feel trapped; is there a way to pass the underscores through to the server, but without the colons?? {short of my going into HTTP::Headers and actually tweaking the code to never "canonicalize"}

      2018-08-13 Athanasius fixed code tag placement

        I didn't understand what you told me about the content/headers stuff.
        That's why I pointed out the distinction between data conveyed in the header and data conveyed in the content/body. IF you want to mimic the form submission, you must not transfer the data in the header - no browser does that! Perhaps, the csrf_token might need to be transferred with the header but that is surely defined in the fine API documentation, right?

        I GUESS, you need something like this:

        #-- everything from the FORM here (nerver use colons here): my@form_fields= ( csrf_token => $csrf_token, #-- above looks right, according to form example subscribees => "$name <$email +>", setmemberopts_btn => 'Submit Your C +hanges', send_notifications_to_list_owner" => "1" ); #-- read the API-doc: IF the csrf_token is required in the header # (use colons to prevent canonicalisation): my @headers = ( ':csrf_token' => $csrf_token, ':member_verbosity_threshold' => 0, 'User-Agent' => 'Mozilla/5.0' ); my $res = $ua->request(POST $_[0], @headers, Content => \@form_fields );
        Regarding the lifetime expiration: Of course, you must provide the currently valid csrf_token. I hope, running this application is in accordance with laws and terms of use...