HTTP::Proxy and X-Forwarded-For headers

cleverett has asked for the wisdom of the Perl Monks concerning the following question:

<input type="hidden" name="perlquestion_doctext" value=1> Hi,

I'm testing an application that runs behind a lightweight Apache as a proxy, and I need to give it different IPs as part of the testing regimen to exercise the Geotargetting part of the application. So to live test the application, I wrote live test harness where I can put different parameters like the referrer url, the remote ip, etc into form widgets, and puts that all into the location of an IFRAME when I submit the form.

Then I used HTTP::Proxy and HTTP::Proxy::HeaderFilter to strip the referer URL and IP address from the query string, and to attach them to the request object as the 'Referer' and 'X-Forwarded-For' headers instead:

package EngineTestFilter;
use base qw/HTTP::Proxy::HeaderFilter/;

use strict;
use warnings;

sub filter {
  my ($self, $headers, $request) = @_;

  my $uri = $request->uri();
  my ($location, $query) = split(/\?/, $uri);
  
  my %params = map { split /=/ } map { split /&/ } $query;

  $request->uri($location  .'?'.join('&', map { "$_=$params{$_}" } qw/
+a s/));
  $headers->header(Referer => $params{referer});
  $headers->remove_header('X-Forwarded-For');
  $headers->header('X-Forwarded-For' => $params{remote_ip});
}
[download]

Unfortunately, the IP address ends up at the beginning of the 'X-forwarded-For' header, and not the end:

  'X-Forwarded-For' => '207.177.71.108, 10.0.0.50, 10.0.0.50',
[download]

I hunted up though the chain of inheritance all the way up to HTTP::Message, and I could not find where the 'X-Forwarded-For' header gets handled in all of that.

Anyone got a cluestick for me?

Comment on HTTP::Proxy and X-Forwarded-For headers Select or Download Code

Replies are listed 'Best First'.
Re: HTTP::Proxy and X-Forwarded-For headers by tachyon (Chancellor) on Sep 11, 2004 at 07:26 UTC
That header: `X-Forwarded-For: client, proxy1, proxy2,.....` [download] is the correct order as each proxy effectively appends to this string. See mod_extract_forwarded for background.... What appears to have happened is that you have successfully removed the XFF header and added in your supplied IP as the client. I expect that the two instances of 10.0.0.50 are succesively added by HTTP::Proxy and then Apache2 after this point. This is expected, as is the order. But this should not matter per se as the client is the leftmost IP (you may choose to ignore anything to the left of a trusted proxy as they could just be spoofing). You could fix the problem simply by adding a mod_extract_forwarded handler and making 10.0.0.50 'trusted', but...... It seems your expectation is that the originating client (for geotargetting) is the right most entity. This is wrong. It is the left most IP in the list. FWIW this snippet comes from some of our code: `my $ip = $ENV{'HTTP_X_FORWARDED_FOR'} \|\| $ENV{'REMOTE_HOST'} \|\| $E +NV{'REMOTE_ADDR'}; # HTTP_X_FORWARDED_FOR may be 'xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx' # originating client is left most IP. this split patches issues as + some proxies # add space after IP and before , ($ip) = split /\s*,/, $ip;` [download] cheers tachyon	[reply] [d/l] [select]
Re^2: HTTP::Proxy and X-Forwarded-For headers by cleverett (Friar) on Sep 13, 2004 at 22:30 UTC
Hmmm, Then what I can't understand is why `Apache->requeqest->connection->remote_ip() returns 10.0.0.50` I'd better look at how I set up mod_rpaf in my apache+mod_perl. If I didn't have it set up, I would expect remote_ip() to return 127.0.0.1	[reply] [d/l]
Re^3: HTTP::Proxy and X-Forwarded-For headers by tachyon (Chancellor) on Sep 13, 2004 at 22:37 UTC
Don't you have this: `Client->HTTP::Proxy 10.0.0.50->Apache 10.0.0.50->Application` [download] The connection is from 10.0.0.50. mod_extract_forwarded fixes that problem. cheers tachyon	[reply] [d/l]
Re^4: HTTP::Proxy and X-Forwarded-For headers by cleverett (Friar) on Sep 13, 2004 at 22:43 UTC
Re^5: HTTP::Proxy and X-Forwarded-For headers by tachyon (Chancellor) on Sep 13, 2004 at 22:49 UTC
Some notes below your chosen depth have not been shown here
Re: HTTP::Proxy and X-Forwarded-For headers by Aristotle (Chancellor) on Sep 11, 2004 at 01:20 UTC
What mock remote IP did you pass the script and what was supposed to show up as the `X-Forwarded-For` header's value? Makeshifts last the longest.	[reply]