in reply to CGI Input with Escaped Characters

Do you have any control over the sending javascript code? If so, then the obvious solution would be to calculate the hash before URI-encoding the data instead of after. If you're really going to be paranoid about making sure the data is transferred correctly, doing it that way would be better anyhow, since it would serve to validate that the data was encoded and decoded correctly, not only that it made it across the wire intact.

Replies are listed 'Best First'.
Re^2: CGI Input with Escaped Characters
by hoyt (Acolyte) on Apr 02, 2017 at 16:45 UTC
    Good point about making sure it gets decoded properly. I do have control over the JS in this instance and started to do that (changing the hash to calculate pre-encoding), but I stopped for three reasons. 1) I encode the string in a JS function piece by piece, then calculate the hash once it's all pieced together. So I'd need to piece together a non-encoded string as well, then hash that. It's a lot of work, and is in a handful of files (pure laziness). JS code snippet below. 2) I'm not sure I'll always have control over the input like that. 3) I was hoping to handle it with Perl for ease. 
        var jData = {request:{'service':'ins_task_queue'},data:[]};
    $('[name=remote_download_id]:checked').each(function(){
        remote_download_id         = $(this).val();
        var here = $(this).siblings();
        var tmpObj = {};
        tmpObj['remote_download_id'] = remote_download_id;
        $.each($(this).siblings(),function(k,v){
-->            tmpObj[here[k].name] = encodeURIComponent(here[k].value
+);
        });
        here = $(this).parent().siblings().children(':input');
        $.each($(this).parent().siblings().children(':input'),function
+(k,v){
-->            tmpObj[here[k].name] = encodeURIComponent(here[k].value
+);
        });
        jData['data'].push(tmpObj);
    });

...

    [download]
    PHP parts: 
        public function getRequest($request,$data,$response_type){
        $reqData['request']['service']     = $request;
        $reqData['data']         = $data;
        $fields = array(
            'data' => json_encode($reqData)
            , 'xyz' => $xyz
            , 'abc' => $abc
        );
        $fields['api_key'] = getApiKey($fields);
        global $debug;
        $debug = $fields;
        return getSvc($fields);
    }

function getApiKey($fields){
    return hash_hmac('sha512',$fields['data'].$fields['abc'].$fields['
+xyz'],$GLOBALS['ses_secret_key']);
}

    [download]
[reply]
[d/l]
[select]
Re^2: CGI Input with Escaped Characters
by hoyt (Acolyte) on Apr 19, 2017 at 01:21 UTC
    I'm still really struggling with this. I've tried to back the calculation up a few steps in the JS code, but that's leading to a ton of other problems for me, so I'm not sure that's going to work too well. I keep coming back to attempting to do this in Perl instead. I've boiled this down to a simple example: 
    #!/usr/bin/perl -w

use strict;
use CGI;
use URI::Escape;
use JSON;
use Digest::SHA qw(hmac_sha512_hex);

my $query         = CGI->new;
my $raw_data     = $query->param('data');
my $data             = decode_json($raw_data);
my $actual_data = '{"request":{"service":"test"},"data":{"test_input":
+"%2B2"}}';

print "raw_data: ".$raw_data."\n";
print "no escaping: ".$data->{data}->{test_input}." vs escaping: ";
print uri_escape($data->{data}->{test_input})."\n";

print hmac_sha512_hex($raw_data,"ABCD1234")."\n";
print hmac_sha512_hex($actual_data,"ABCD1234")."\n";

    [download]
    If you run that on the command line like this: ./script.pl 'data={"request":{"service":"test"},"data":{"test_input":"%2B2"}}' You'll get this: 
    raw_data: {"request":{"service":"test"},"data":{"test_input":"+2"}}
no escaping: +2 vs escaping: %2B2
3c6de296682e7f3896073fe41af9732a294ef723bb1e5c75aa1eba1af981f04f0a0963
+d03604119ea92b719a2912ef0c957c03a7268b51e2170f8fed7c875465
32595bf215b309a73c8dd4d09600430378f455c7cb44d31573b08566ddff0a7bd3c536
+8d70696b57a2c1c95e862ed7b062501e39820bf973c9309812250df460

    [download]
    I need the 32595... calculation to compare to the input (which I removed from the Perl example to make it shorter). I can't just unescape the whole string, because then it'll attempt to escape the part that makes it a JSON input ({, :, etc). It's as if the CGI input unescapes automatically and I can't figure out how to make it not unescape (or "re-escape"). Any ideas on how to do that before I write something myself to deal with escaped characters in a hmac sha? Thanks!
[reply]
[d/l]
[select]

      Hi,

      You forgot to escape  {"request":{"service":"test"},"data":{"test_input":"%2B2"}}

      This is an error  CGI->new( 'data={"request":{"service":"test"},"data":{"test_input":"%2B2"}}' );

      You want this  CGI->new( 'data=%7B%22request%22%3A%7B%22service%22%3A%22test%22%7D%2C%22data%22%3A%7B%22test_input%22%3A%22%252B2%22%7D%7D')

      You want this  CGI->new( { data => $actual_data, } )

      Its like writing 

      my $query = CGI->new({});
$query->param( 'data', $actual_data  );
print $query->self_url;

      [download]

      This is how query-string/form-data works

      Now if you were to use HTTP POST/PUT/PATCH you could use raw json in the request ; they call that AJAX 

      {
    use LWP;
    my $req = HTTP::Request->new(
        POST => 'http://127.0.0.1:80/'
    );
    $req->content_type( 'application/json' );
#~     $req->header('X-File-Name' =>  'tiny.gif' );
    $req->content_length( length $actual_data );
    $req->content( $actual_data );
    print "\n", $req->as_string, "\n";
}
__END__
POST http://127.0.0.1:80/
Content-Length: 59
Content-Type: application/json

{"request":{"service":"test"},"data":{"test_input":"%2B2"}}

      [download]

      And to get at this ajax request from a .cgi you'd use 

      use CGI 4.35;
my $raw_data = CGI->new->param('POSTDATA'); # or PUTDATA or PATCHDATA

      [download]

      An "fundvanced" example Mojolicious::Lite +and jQuery +AJAX + Mojo::Template

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom