c-era has asked for the wisdom of the Perl Monks concerning the following question:

I have an interesting problem. We have a client that wants to use webmethods to send us xml data via an http post. I've tried using cgi.pm, but haven't had much success with it. I've rolled my own (I can see the steam rising from merlyn). First, I'd like to know if I can get cgi.pm to work properly for me. If I can't, I'd like any input with potential problems with my code.

This is the cgi script I wrote. It needs to take the content and not parse, or decode anything and write it out to a file. 

use strict;
use CGI;
if ($ENV{'CONTENT_LENGTH'} > 0 && $ENV{'REQUEST_METHOD'} eq "POST" && 
+$ENV{"CONTENT_TYPE"} =~ m|^text/xml|){
    my $buffer;
    read(STDIN, $buffer,$ENV{'CONTENT_LENGTH'});
    if (length($buffer) != $ENV{'CONTENT_LENGTH'}){
        # code to return 400
        print "400\n";
        exit;
    }
    open (FILE,">d:\\test\\test");
    print FILE $buffer;
    close (FILE);
    # code to return 200
    print "200\n";
    exit;
}
# code to return 400
print "400\n";
exit;

[download]

Here is a test agent program I wrote that mimics how they will send us data: 

use LWP::UserAgent;
use strict;

my $ua = new LWP::UserAgent;
my $content = <<EOF;
<test>
    <test2>test4&;</test2>
<test>
EOF
my $req = new HTTP::Request (POST=>'http://test:test@localhost/cgi-bin
+/http/upload.pl');
$req->content_type('text/xml');
$req->content("$content");
print $ua->request($req)->as_string;

[download]

Replies are listed 'Best First'.
Re: CGI.pm problems
by Fletch (Bishop) on Dec 13, 2001 at 22:21 UTC
[reply]
Re: CGI.pm problems
by xunker (Beadle) on Dec 13, 2001 at 22:12 UTC

    I've been using XML posting through LWP for a couple of projects and another (albeit less elegant but much easier) way around it is to let it be a regular HTTP post and pull the xml from from CGI::param (i.e. $query->param ('xml');).

    Another thing I noticed: you 'use CGI;', but you never actually call it anywhere in the code, right? Could just the using of CGI.pm be causing some of your lossage?

[reply]
      The problem with using $query->param('xml') is, they can't send an xml= infront of the xml, the second problem is &; will appear frequently in the xml (which is where I'm losing the data with cgi.pm).

      As for the use CGI; I just didn't remove it from the example. It makes no difference if I remove it.

[reply]
[d/l]
[select]
        According to a reference here, the ampersand must be converted into an entity in element content. Change that to &amp;; and it'll be more valid. (Think of it this way -- if bare ampersands were allowed, how would you be able to find entities?)

        I suspect that may have a dramatic effect on how CGI.pm works.

[reply]
[d/l]
        As for the use CGI; I just didn't remove it from the example. It makes no difference if I remove it.

        So this is the "roll-your-own" version? Why didn't you post the CGI.pm version that you're having trouble with? If you want help fixing your CGI.pm code, you will have to actually post it for people to look at.

[reply]
Re: CGI.pm problems
by sevensven (Pilgrim) on Dec 14, 2001 at 01:16 UTC

    Yes, you can get CGI.pm to work for you.

    You just need to unescape the xml that you receive from the post, like this : 

    #!/usr/bin/perl
use CGI qw/unescape/;
use strict;

my $cgi = new CGI;

print $cgi->header;
my $xml = $cgi->param('xml');
print unescape($xml);

    [download]

    And your previous, own rolled, rigthly merlin steaming version ;^) would work if you had not forgoten the proverbial print "Content-type: text/html\n\n"; before trying to output the https error codes.

    -- sevensven or nana korobi, ya oki
[reply]
[d/l]
[select]
Re: CGI.pm problems
by chip (Curate) on Dec 14, 2001 at 01:47 UTC
    File upload is probably the most reliable way to handle this. Look in perldoc CGI for "CREATING A FILE UPLOAD FIELD".

        -- Chip Salzenberg, Free-Floating Agent of Chaos

[reply]
[d/l]
Re: CGI.pm problems
by Illiad (Initiate) on Dec 14, 2001 at 04:07 UTC
    Rather simple reason that CGI.pm isn't doing what you'd like it to:

    Your incomming XML is likely unquoted.

    Let's assume you're trying to pass in this small XML string (I'll do the example in GET, since CGI.pm handles GET and POST transparantly): "<xmltag>Tulips &amp; Flowers<xmltag>".

    in a get request this is going to look like:
    script.pl?xml=<xmltag>Tulips &amp; Flowers<xmltag>

    Ick! Since you haven't followed the specs for escapeing URLs, it's going to get nasty.

    Get the people who are sending your request to properly escape the data: 

    my $newval = CGI::escape(qq("<xmltag>tulips &amp; Flowers:</xmltag>));

    [download]
    $newval will now look more like: %3Cxmltag%3Etulips%20%26amp%3B%20Flowers%3A%3C%2Fxmltag%3E

    Which is how it should look if you're following the standards for URL quoting.

    Now, using CGI.pm: 

    use CGI.pm

my $q = new CGI;

my $xml = CGI::unescape($q->param('xml'));

    [download]

    Voila!

[reply]
[d/l]
[select]
Re: CGI.pm problems
by CMonster (Scribe) on Dec 14, 2001 at 06:35 UTC

    I've noticed a common misperception here. If this XML is being sent the way I think it is, it's not URL encoded at all. In your example LWP script, for instance, it's being POSTed as text/xml, not text/url-encoded (or whatever).

    Luckily, since the message is being sent raw you don't have to worry much about escaping/unescaping or similar.

    This is common for SOAP-type messages as well. The answer is to roll your own (if it's raw XML) or use SOAP::Lite (if it's SOAP.) I did a bit of each for my own server. I can post examples if you wish.

    Now merlyn can flame me. :)

    ~chris

[reply]
      Ah, someone else who works in the real world... You're right about the misperception, I was about to post another question (maybe I still will) to clear it up.
[reply]

Back to Seekers of Perl Wisdom