Viewing locally-served pages from the outside, while *inside*!

Or, "My home webserver thought it was ugly until it looked in the mirror."

Over a year ago, I had a situation where I wanted to see what my locally-served website looked like from the "outside." This was in a drive to configure my home network and webserver to correctly serve pages to anyone I send my IP address, what with all the various configuration issues facing me at the time. My home network consists of an internal LAN fed by a Linksys cable-modem router with ports 80 and 8080 forwarded to my local webserver.

"But," you say, "why don't you just use the local internal IP of your webserver to see your site?"

"Well," I reply, "I want to make sure people on the outside are getting the same thing I'm getting from the inside with no surprises. Also, from the inside, if I use my router's external IP to view my site, I get the 401/403 login challenge on my router's web administration panel."

"Hrumph."

Hrumph indeed. So, everyone with me: "Perl to the rescue!"

Problem analysis:

What I needed was a script that resided on my externally-hosted and paid-for domain host that I could use to reflect an HTTP request. I thought of methods to make it call only my home's IP (all of them smelly, mangled, and snarled), but I settled on making it general enough to be used from any system that had a running webserver. Hey, what a neat idea!

Wanna try? If you have an externally-accessible server that answers to port 8080 on your externally-visible IP address (I had to cludge to circumvent my provider's "port 80 filter"...) then take a look at your site here. You will, in theory, get your locally-hosted site as seen from my hosted site. (Actual mileage may vary. Expect requests from phaysis.com in your access logs.)

Configuration:

On my hosted site, I have a subdirectory set aside in my document root which contains the mirror.cgi script and an .htaccess config file (your host must allow .htaccess files for this to work).

Further documentation assumes your mirror subdirectory is called /mirror .

.htaccess:

The .htaccess file contents are as follows:

DirectoryIndex mirror.cgi
ErrorDocument 404 /mirror/mirror.cgi
[download]

This tells Apache to serve mirror.cgi by default, which then executes to grab the root page on your local site. For any file on your local site which (surprise surprise) doesn't exist in the remote /mirror directory, Apache's ErrorDocument functions call your script to take care of file fetching.

mirror.cgi:

Ok, so here's the meat and potatos of our operation. What we're doing is creating a LWP::UserAgent instance to get files from the IP of the computer making the request. Currently, the code kind of looks inelegant (my apologies) but it has worked very well for me in its 1 1/2 years of existence (further proving Aristotle's suggestion that "Makeshifts last the longest").

The code for mirror.cgi is as follows:

#!/usr/bin/perl

# mirror.cgi
# gets the ip of the calling client, and sends an http request
# to that client. This is used to browse the client's local web
# server while getting around the client's router, in effect
# showing the client how their locally-served site functions
# from the outside.
# Does not support sending post variables, yet.

use LWP::UserAgent;

########## Handle the request and build a reflection request:

##### Get the client address
$clientip = $ENV{'REMOTE_ADDR'};
##### Get the requested URL
$req_url = ($ENV{'REDIRECT_URL'}) ? $ENV{'REDIRECT_URL'} : '';
##### Get rid of the leading directory on the local call
$req_url =~ s/^\/mirror//gi;
##### Default to 8080, to bypass ISP's 80-filter
$serverport = ":8080";
##### Build the final URI/URL
$geturl = "http://".$clientip.$serverport.$req_url;

########## Create a new user agent
$ua = LWP::UserAgent->new(      env_proxy => 1,
                                keep_alive => 1,
                                timeout => 30
      );

########## Create a new request
$request = HTTP::Request->new('GET', $geturl);

########## Get the requested page with the new user agent
$resultref = $ua->request($request);

########## Print the results

##### Print the returned headers
print $resultref->{'_headers'}->as_string();
##### Print blank line to end headers and begin document entity
print "\n";
##### And here's the body
print $resultref->content;

##########
exit;
##########
[download]

Hopefully, the code is simple and straightforward enough to warrant not much further explanation. It's a "straight wire" of a script, but it's just as effective. One caveat, though: this script doesn't handle POSTed form variables, only HTTP GET requests. I've considered extending to cover this, but currently there's little need.

Problem solved:

So, now all I have to do to see my internal site as though I was outside is to, in my example, request the page http://www.phaysis.com/mirror/, and voila, my server's hard drive comes to life as it serves a page request for my root index file. As a bonus, and much to my amazement, the script even handles images with no problems thanks to the flexibility of LWP's methods.

I hope this is of use to some of you. It certainly has been useful to me in testing scripting and design of my local site (which is quicker and easier to update than my hosted space), as well as testing external access issues such as ISP's that filter what their paying customers can do, router port-forwarding issues, and local LAN connectivity issues. Very useful.

Heh, and just remember to ignore all the "404 /mirror/..." messages in your hosted domain's error_log files. Those'll get you.

ENJOY!

-Shawn / (P^h) Phaysis
If idle hands are the tools of the devil, are idol tools the hands of god?
update: fixed some typos and some formatting issues - Phaysis

Comment on Viewing locally-served pages from the outside, while inside! Select or Download Code

Replies are listed 'Best First'.
Re: Viewing locally-served pages from the outside, while inside! by SysApe9000 (Acolyte) on Sep 29, 2002 at 23:44 UTC
I think your perl solution to this problem is neat... but there is a more general solution that doesn't involve perl, and is easier to implement. Is there a server on the internet you can ssh into? Then chances are, you only need a single invocation of ssh to accomplish the above, and it can be used for all sorts of other situations. Let's assume that mybox is the address of the machine you want to test out. Assume that otherbox is an ssh-able server on the internet somewhere. Try something like: ssh -L 9080:mybox:80 otherbox This will open a port on your machine (9080) which when connected to will cause that data to be sent (encrypted, in this case probably not neccesary, but whatever) to otherbox. Otherbox will connect to mybox on port 80, forwarding that information through. From the perspective of mybox, the connection will be no different from any connection attempt coming from that machine over the internet. In other words, run the above ssh command and then point your browser to http://localhost:9080/ and you will be connected to mybox, but indirectly via otherbox! Sorry if my explanation isn't all that clear, ssh's port forwarding features are really handy but it can be a bit tricky to wrap your head around what all is possible with them.	[reply]
Re: Viewing locally-served pages from the outside, while inside! by rjimlad (Acolyte) on Sep 29, 2002 at 14:35 UTC
There are actually a bunch of similar tricks you can do in apache to persuade it to always run a particular script for any requests, perhaps the most sensible of which is to use mod_rewrite to rewrite everything to your script (and thus log everything as 200s not 404s). However if you really want to tell it to always run that script, you should also be able to do so in an .htaccess file this way: `Action my-mirror /mirror/mirror.cgi SetHandler my-mirror` [download] ...and have another .htaccess file in /mirror/ saying: `SetHandler cgi-script` [download] ...or mod_perl as appropriate. There may be a shorter way, but I'm not in a position to test it right now. (Edited to fix (english) grammar. Oops) (Edited to remove inintentionally humourous english link)	[reply] [d/l] [select]
Re: Re: Viewing locally-served pages from the outside, while inside! by Phaysis (Pilgrim) on Sep 29, 2002 at 20:53 UTC
Hey, thanks for the tip. ++! I was hoping to figure out how to not make all those 404's appear in my logs. mod_rewrite...Hmm. I've heard of it, but haven't done that much research into it. I'm reading up on it now, and I think I'm starting to like it. Not completely understanding it, but I like it. Cheers! -Shawn / (P^h) Phaysis If idle hands are the tools of the devil, are idol tools the hands of god?	[reply]
Re: Viewing locally-served pages from the outside, while inside! by Anonymous Monk on Oct 04, 2002 at 07:31 UTC
Since your router is forwarding port 8080-> 80, couldn't you just access (from the inside) your.external.ip.number:8080? When I do that from the inside, I don't get the 401/403 challenge that I get when accessing port 80 (either implicitly or explicitly). I have a linksys 4-port router, so your mileage may differ	[reply]
Re: Viewing locally-served pages from the outside, while inside! by satanklawz (Beadle) on Oct 02, 2002 at 03:22 UTC
Why not just use an external proxy server? Tho I DO like your solution. :)	[reply]
Re: Re: Viewing locally-served pages from the outside, while inside! by Phaysis (Pilgrim) on Oct 03, 2002 at 01:50 UTC
>Why not just use an external proxy server? Hmm. Well, though your suggestion, as well as SysApe9000's suggestion, is excellent and much more robust, my problem is that outside of my home LAN, the only server presence, the only control I have is my paid-for webhost which, sadly, does not allow SSH logins and, since I am on a shared server, does not allow me to set up and configure a proxy. Sad, really. Thank you folks for your input and feedback. This script has helped me much; I'm already getting fun ideas for network programming in Perl. I look forward to playing with your suggestions sometime in my future, just not right now. ::sighs:: -Shawn / (P^h) Phaysis If idle hands are the tools of the devil, are idol tools the hands of god?	[reply]