http://qs1969.pair.com?node_id=194558

davorg has asked for the wisdom of the Perl Monks concerning the following question:

I'm having some problems with some LWP that I'd appreciate some help with. I'll describe the problem, my proposed solution and what's wrong with my solution.

The Problem

I'm collecting data from a form and passing it on to a third party CGI program. The data inputs on the form are determined by what is sensible to give the user. The data passed on to the CGI program is determined by the owners of the program.

These two sets of requirements are at odds with each other.

For example, I'm giving customers a choice of paying £x/month or £y/year. This is obviously going to be controlled by one input (either a select with two options or a pair of radio buttons). It makes no sense to have one input for "amount" and another for "period". Unfortunately this is exactly what the CGI program expects.

So my problem is that I need to send a different set of CGI parameters to the ones that are on the form.

My Solution

So I thought I'd write another CGI program that sits between the form and the third-party CGI program. My program would look at the incoming parameters, calculate any extra parameters needed (e.g. create an "amount" parameter based on the value in the "period" parameter), and then make a POST request to the external CGI program. It then need to display the results it gets back in the browser.

Here's roughly what I've got.

#!/usr/bin/perl -w use strict; use CGI ':standard'; use LWP; my $url = 'http://blah, blah, blah'; my $ua = LWP::UserAgent->new; $ua->cookie_jar( {} ); # Note: assumes no multivalued parameters my %param = map { $_ => param($_) } param; # removed code to munge %param my $resp = $ua->post($url, [ %param ]); print header; print $resp->content;

This pretty much does what I want, but there are a couple of problems.

The Problems

  1. The browser shows the URL of my progrma rather than the external CGI program.
  2. The relative URLs within the displayed content are all broken as the browser is interpreting them as relative to my program.

It seems to me that what I really want is a "redirect", but you can't redirect a POST request (or can you?)

Are there any other suggestions? Am I missing something obvious?

--
<http://www.dave.org.uk>

"The first rule of Perl club is you do not talk about Perl club."
-- Chip Salzenberg

Replies are listed 'Best First'.
Re: LWP Redirection Problem
by Kanji (Parson) on Sep 02, 2002 at 10:52 UTC

    It seems to me that what I really want is a "redirect", but you can't redirect a POST request (or can you?)

    According to the spec it's possible, but in practice I've found the opposite to be true (see the caveat in the aforementioned link); meaning you're out of luck unless the external CGI can be coaxed into accepting GET requests.

    Some alternatives to consider, though...

    1. The browser shows the URL of my progrma rather than the external CGI program.

    Skip a Perl intermediary altogether, and use JavaScript to do your param munging with an onClick/onSubmit.

    1. The relative URLs within the displayed content are all broken as the browser is interpreting them as relative to my program.

    Have your script include a <base href="http://url.to/the/external/CGI" /> in it's output so that relative URLs are correctly interpreted by thr browser.

        --k.


      Some alternatives to consider, though...
      1. The browser shows the URL of my progrma rather than the external CGI program.
      Skip a Perl intermediary altogether, and use JavaScript to do your param munging with an onClick/onSubmit.

      Sorry. not really an option. Many people (very sensibly) diable JavaScript.

      2. The relative URLs within the displayed content are all broken as the browser is interpreting them as relative to my program.
      Have your script include a <base href="http://url.to/the/external/CGI" /> in it's output so that relative URLs are correctly interpreted by thr browser.

      As I understand it, a "Content-Base:" header will work just as well and will be easier to add. I'm looking at that option now.

      --
      <http://www.dave.org.uk>

      "The first rule of Perl club is you do not talk about Perl club."
      -- Chip Salzenberg

Re: LWP Redirection Problem
by Arien (Pilgrim) on Sep 02, 2002 at 11:02 UTC

    There are two options:

    1. Stick a CGI in between (as you described) and use <base href="$FOO"> or replace any relative URLs in the response from the remote server with absolute URLs. Your client-side problem remains: the URL shown will be that of the CGI in between.
    2. Do the "form fixing" on the client-side and submit directly to the remote server. You now have a client-side scripting problem.

    You can in theory redirect a POST request but IIRC most browsers will do a GET on the new resource. Also, were it to work, you'd still be at square one: you'd be submitting the same data you were submitting before, which was not suitable for consumption by the remote CGI (which is where it's now being sent).

    — Arien

Re: LWP Redirection Problem
by thpfft (Chaplain) on Sep 02, 2002 at 11:27 UTC

    This is a rather low-tech solution, but one that I've used several times where incompatible systems needed to be strung together: invent a spurious confirmation page.

    Eg. instead of exposing people to the execrable interface offered by the credit card handler that amnesty uses, I've created a set of (gratingly cheerful) forms which culminate in a 'please check and continue' page, the sole purpose of which is to assemble a hidden form which presents the data to the remote system.

    (As with any such intervention - including your LWP approach - there's still the problem of tracking changes in the remote system.)

    .2p

Re: LWP Redirection Problem
by fruiture (Curate) on Sep 02, 2002 at 10:49 UTC

    Well, you could at least try to place a <base href="..."> in the head of the incoming document to make all relaive URLs work.

    POST-Redirecting is not possible (i just had a look into RFC2616).

    --
    http://fruiture.de
Re: LWP Redirection Problem
by arhuman (Vicar) on Sep 02, 2002 at 20:49 UTC
    POST redirection is possible.
    It's even NECESSERY if you want to post on some (IIS) server poorly configured,
    despite what the RFC says.
    (we had to do it several times at the office when multi-posting on the web...)

    In my memories it involved handling a 'redirect_ok'(!?!) method,
    but a quick look at the version installed on my box
    leads me to rather say that's now(?) the answer is :'UserAgent->requests_redirectable'.
    (anyway you might want to see this old post)


    "Only Bad Coders Code Badly In Perl" (OBC2BIP)
Re: LWP Redirection Problem
by zaimoni (Beadle) on Sep 03, 2002 at 03:04 UTC

    Let's see....

    1. The browser shows the URL of my program rather than the external CGI program.

    Fake it. That's what the <title> tag is for. Try

    <title>WantThisURLInSight</title>

    2. The relative URLs within the displayed content are all broken as the browser is interpreting them as relative to my program.

    As several other posts have already explained in detail, the standard HTML for this uses the <base> tag.

      <title> won't affect what the browser displays in its address bar.

      Makeshifts last the longest.

        True. However, the address bar can be removed with JavaScript: tell

        window.open(URL,WindowName,ConfigString)

        to delete it with

        location=no

        in the ConfigString. Of course, there are several other parameters to set location to that have equivalent effect.