in reply to POST fieldnames with underscores

For HTTP-headers there seems to be a conversion. From HTTP::Headers (used by HTTP::Request::Common via HTTP::Message):

ON-CANONICALIZED FIELD NAMES
The header field name spelling is normally canonicalized including the '_' to '-' translation. There are some application where this is not appropriate. Prefixing field names with ':' allow you to force a specific spelling. For example if you really want a header field name to show up as "foo_bar" instead of "Foo-Bar", you might set it like this:

$h->header(":foo_bar" => 1);

Assuming, you want to post a list of variables, do transfer them as content (application/x-www-form-urlencoded), not as HTTP-headers.
Please check POST method invocation parameters (order/names) or post the specific lines, where you're creating the POST request, for further analysis.

Replies are listed 'Best First'.
Re^2: POST fieldnames with underscores
by BernieC (Pilgrim) on Aug 11, 2018 at 23:30 UTC
    No wonder I couldn't find it...:o). For example 
    <FORM action="../../../admin.cgi/testlist-fantasyfarm.com/privacy/send
+er" method="POST" >
<input type="hidden" name="csrf_token" value="280200000069096e6f5b7347
+000000746573746c6973742d66616e746173796661726d2e636f6d2b61646d696e3a6
+638333163393562386564343865383939386464623764346337663661356161333937
+6135316132">

    [download]
    and so I modified my program to stick a ':" in front of every field and Just as you said - -I tried it and it program *worked*. 
    'user-agent' => 'Mozilla/5.0',
':member_verbosity_threshold' => 0,
':csrf_token' => '28020000006916706f5b7347000000746573746c6973742d6661
+6e746173796661726d2e636f6d2b61646d696e3a63356234316237373565643432393
+036383931306131616635643239653666366463643239373065',
'content-type' => 'application/x-www-form-urlencoded',

    [download]
    Thanks! I'd have never found that info {since I had no idea what I should be looking for}.
[reply]
[d/l]
[select]

      Good, it works for you, but I am not convinced, your program works as intended. Consider:

      Token in HTTP-header (your current fix): 

      my $res = $ua->request(POST $url,
                       'user-agent'                  => 'Mozilla/5.0',
                       ':member_verbosity_threshold' => 0,
                       ':csrf_token'                 => '2802000000...
+',
#-- not required:      'content-type'                => 'application/x
+-www-form-urlencoded',
                      );

      [download]

      Result: 

      POST /xxx HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: localhost:5000
User-Agent: Mozilla/5.0
Content-Length: 0
Content-Type: application/x-www-form-urlencoded
csrf_token: 28020000006.....
member_verbosity_threshold: 0

      [download]

      Token in HTTP-body (content) - this is more like what your <FORM ...> example would do: 

       my $res2 = $ua->request(POST $url,
                       'user-agent'                  => 'Mozilla/5.0',
                       ':member_verbosity_threshold' => 0,
#-- not in header:     ':csrf_token'                 => '28020000006..
+...',
                       Content                       => [ csrf_token =
+> '28020000006.....', ]  #-- but in body
                      );

      [download]

      Result: 

      POST /xxx HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: localhost:5000
User-Agent: Mozilla/5.0
Content-Length: 27
Content-Type: application/x-www-form-urlencoded
member_verbosity_threshold: 0

csrf_token=28020000006.....

      [download]

      Hope, that shed some light. I was also going to nitpick, that you should prepend your non-standard headers with 'X-' (like X-Csrf-Token), but that practise seems discouraged since a couple of years: SO: Custom HTTP headers : naming conventions. Thanks, I learned something new ;-)

[reply]
[d/l]
[select]
        OK, I'm real confused now. I notice that my form has, in fact, stopped working. to clarify: I only noticed that there were underscores -- I didn't notice that the colons were still there and that, in fact, the program didn't work}. My guess is that the server doesn't like the colons.. :o) {As for the X-, not my choice: I simply scrape the form I get and try to return the right fields}. I didn't understand what you told me about the content/headers stuff. Here is {part of the} actual form I fetched 
        <FORM action="...members/add" method="POST" enctype="multipart/form-da
+ta">
<input type="hidden" name="csrf_token" value="280200000069c97e705b7347
+000000746573746c6973742d66616e746173796661726d2e636f6d2b61646d696e3a3
+763663438346436393036323038363738613566383231623461323132353762356361
+3763633665">
[...]
<INPUT name="send_notifications_to_list_owner" type="RADIO" value="0" 
+>
 <INPUT name="send_notifications_to_list_owner" type="RADIO" value="1"
+ CHECKED >
  <TEXTAREA NAME=subscribees ROWS=10 COLS=70%></TEXTAREA>
[...]
<INPUT name="setmemberopts_btn" type="SUBMIT" value="Submit Your Chang
+es" >

        [download]
        Then what I do is set up a @fields array with 
        @fields= (    ":csrf_token" => $csrf_token
                    subscribees => "$name <$email>",
               ":setmemberopts_btn" => 'Submit Your Changes',
               ":send_notifications_to_list_owner" => "1",
               );
$res = $ua->request(POST $_[0], \@fields);

        [download]
        and that not only doesn't work, but gets me an ugly 
        <h3><strong><font color="red" size="+2">Error: </font></strong><em>The
+ form lifetime has expired. (request forgery check)</em></h3><center>
+<h2>Testlist mailing list administration<br>Membership&nbsp;Managemen
+t... Section</h2></center><hr>

        [download]
        I'm not sure what they check for the 'lifetime' but I bet it is the csrf_token. and I see in the request that every field name now has a colon in front of it {except for the fields that showed up in the request-content; they all have %3's in front of them}. So I feel trapped; is there a way to pass the underscores through to the server, but without the colons?? {short of my going into HTTP::Headers and actually tweaking the code to never "canonicalize"}

        2018-08-13 Athanasius fixed code tag placement

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom