Good point about making sure it gets decoded properly. I do have control over the JS in this instance and started to do that (changing the hash to calculate pre-encoding), but I stopped for three reasons. 1) I encode the string in a JS function piece by piece, then calculate the hash once it's all pieced together. So I'd need to piece together a non-encoded string as well, then hash that. It's a lot of work, and is in a handful of files (pure laziness). JS code snippet below. 2) I'm not sure I'll always have control over the input like that. 3) I was hoping to handle it with Perl for ease.

    var jData = {request:{'service':'ins_task_queue'},data:[]};
    $('[name=remote_download_id]:checked').each(function(){
        remote_download_id         = $(this).val();
        var here = $(this).siblings();
        var tmpObj = {};
        tmpObj['remote_download_id'] = remote_download_id;
        $.each($(this).siblings(),function(k,v){
-->            tmpObj[here[k].name] = encodeURIComponent(here[k].value
+);
        });
        here = $(this).parent().siblings().children(':input');
        $.each($(this).parent().siblings().children(':input'),function
+(k,v){
-->            tmpObj[here[k].name] = encodeURIComponent(here[k].value
+);
        });
        jData['data'].push(tmpObj);
    });

...
[download]

PHP parts:

    public function getRequest($request,$data,$response_type){
        $reqData['request']['service']     = $request;
        $reqData['data']         = $data;
        $fields = array(
            'data' => json_encode($reqData)
            , 'xyz' => $xyz
            , 'abc' => $abc
        );
        $fields['api_key'] = getApiKey($fields);
        global $debug;
        $debug = $fields;
        return getSvc($fields);
    }

function getApiKey($fields){
    return hash_hmac('sha512',$fields['data'].$fields['abc'].$fields['
+xyz'],$GLOBALS['ses_secret_key']);
}
[download]

I'm still really struggling with this. I've tried to back the calculation up a few steps in the JS code, but that's leading to a ton of other problems for me, so I'm not sure that's going to work too well. I keep coming back to attempting to do this in Perl instead. I've boiled this down to a simple example:

#!/usr/bin/perl -w

use strict;
use CGI;
use URI::Escape;
use JSON;
use Digest::SHA qw(hmac_sha512_hex);

my $query         = CGI->new;
my $raw_data     = $query->param('data');
my $data             = decode_json($raw_data);
my $actual_data = '{"request":{"service":"test"},"data":{"test_input":
+"%2B2"}}';

print "raw_data: ".$raw_data."\n";
print "no escaping: ".$data->{data}->{test_input}." vs escaping: ";
print uri_escape($data->{data}->{test_input})."\n";

print hmac_sha512_hex($raw_data,"ABCD1234")."\n";
print hmac_sha512_hex($actual_data,"ABCD1234")."\n";
[download]

If you run that on the command line like this: ./script.pl 'data={"request":{"service":"test"},"data":{"test_input":"%2B2"}}' You'll get this:

raw_data: {"request":{"service":"test"},"data":{"test_input":"+2"}}
no escaping: +2 vs escaping: %2B2
3c6de296682e7f3896073fe41af9732a294ef723bb1e5c75aa1eba1af981f04f0a0963
+d03604119ea92b719a2912ef0c957c03a7268b51e2170f8fed7c875465
32595bf215b309a73c8dd4d09600430378f455c7cb44d31573b08566ddff0a7bd3c536
+8d70696b57a2c1c95e862ed7b062501e39820bf973c9309812250df460
[download]

I need the 32595... calculation to compare to the input (which I removed from the Perl example to make it shorter). I can't just unescape the whole string, because then it'll attempt to escape the part that makes it a JSON input ({, :, etc). It's as if the CGI input unescapes automatically and I can't figure out how to make it not unescape (or "re-escape"). Any ideas on how to do that before I write something myself to deal with escaped characters in a hmac sha? Thanks!

Hi,

You forgot to escape  {"request":{"service":"test"},"data":{"test_input":"%2B2"}}

This is an error  CGI->new( 'data={"request":{"service":"test"},"data":{"test_input":"%2B2"}}' );

You want this  CGI->new( 'data=%7B%22request%22%3A%7B%22service%22%3A%22test%22%7D%2C%22data%22%3A%7B%22test_input%22%3A%22%252B2%22%7D%7D')

You want this  CGI->new( { data => $actual_data, } )

Its like writing

my $query = CGI->new({});
$query->param( 'data', $actual_data  );
print $query->self_url;
[download]

This is how query-string/form-data works

Now if you were to use HTTP POST/PUT/PATCH you could use raw json in the request ; they call that AJAX

{
    use LWP;
    my $req = HTTP::Request->new(
        POST => 'http://127.0.0.1:80/'
    );
    $req->content_type( 'application/json' );
#~     $req->header('X-File-Name' =>  'tiny.gif' );
    $req->content_length( length $actual_data );
    $req->content( $actual_data );
    print "\n", $req->as_string, "\n";
}
__END__
POST http://127.0.0.1:80/
Content-Length: 59
Content-Type: application/json

{"request":{"service":"test"},"data":{"test_input":"%2B2"}}
[download]

And to get at this ajax request from a .cgi you'd use

use CGI 4.35;
my $raw_data = CGI->new->param('POSTDATA'); # or PUTDATA or PATCHDATA
[download]

An "fundvanced" example Mojolicious::Lite +and jQuery +AJAX + Mojo::Template

If it is a GET query, then the original data should be in $ENV{QUERY_STRING} iirc.

That looks correct, and may work, but I try to keep the Perl code so that it'll work from command line too for troubleshooting. When I output the $ENV{} from command line, it doesn't show to that as an available key. Would that only work from a web posting action? Thanks!

You can pass CGI parameters on the command, using data=foo api_key=1234. See the DEBUGGING section of CGI. But I doubt that CGI will shoehorn that back into %ENV.

You could set the environment variable on the command line also

$ env QUERY_STRING=data=foo&api_key=laurel%26hardy script.cgi
[download]

or you could check inside your script if it is run from the commandline:

if (-t STDIN) {
    # command line. for params, substitute special chars (e.g. & => %2
+6)
    # (there must be a module out there which does that for you)
} else {
    # via webserver
}
[download]

perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

How are you seeing that it is being sent as %26 ?

I echo out the string in a JS console in my browser when troubleshooting. If I take that whole string to an hmac generator site, I can tell that the hash matches with %26, and if I print it from the Perl command line, it matches the hash from an & sign.

It seems like you need to convert & back to %26

my $raw_data =  uri_escape $query->param('data'); # not uri_unescape

poj