webadept has asked for the wisdom of the Perl Monks concerning the following question:

Hi all you wonderful monks!

There are five different types of cloaking:
1) User Agent Cloaking (UA Cloaking)
2) IP Agent Cloaking (IP Cloaking)
3) IP and User Agent Cloaking (IPUA Cloaking).
4) Referral based cloaking.
5) Session based cloaking

Now, User Agent Cloaking is really easy to detect. I have no problem there. Tell the page that you are a IE 6.0 on the first check and the Googlebot/1.2 on the second and see what it gives you. No problem...

What I don't know how to do is to send a different IP to get around the IP Agent Cloaking. Does anyone know a way of doing this with LWP or some other mod? I have no limits on what I can use, but can't seem to find a good way to do this. I only need to tell the Webserver that I'm the Googlebot, doesn't need to be any tricky, get into the internal network, garbage. Just the hit to the Webserver, so if they are running a cloaked page they believe me enough to send me what they would send the Googlebot.

Any ideas?

webadept

Every day someone is doing what someone else said is impossible.

Replies are listed 'Best First'.
Re: Checking for Cloaked Webpages
by Fletch (Bishop) on Dec 06, 2002 at 18:19 UTC

    First you need to hook up the gas anomaly detector to the photon torpedo's guidance system . . . . Wait, that's a cloaked Bird of Prey.

    Presuming you control enough of the network infrastructure between you and the webserver so that you can succesfully spoof whatever IP and get the traffic back to you, if you setup an interface with the right IP you can fairly easily tell LWP to use that address for sending outgoing traffic. 

    package MyHTTP; 
use base qw(LWP::Protocol::http); 

sub _new_socket { 
    my($self, $host, $port, $timeout) = @_;

    local($^W) = 0;  # IO::Socket::INET can be noisy
    my @args = (PeerAddr => $host,
                PeerPort => $port,
                Proto    => 'tcp',
                Timeout  => $timeout,
           );
    push @args, LocalAddr => $ENV{LWP_LOCAL_ADDR}
    if exists $ENV{LWP_LOCAL_ADDR};

    my $sock = IO::Socket::INET->new( @args );

    unless ($sock) {
        # IO::Socket::INET leaves additional error messages in $@
        $@ =~ s/^.*?: //;
        die "Can't connect to $host:$port ($@)";
    }
    $sock;
}

LWP::Protocol::implementor( 'http', 'MyHTTP' );

package main;

use LWP::Simple qw( getprint );

getprint( shift || 'http://localhost/' );

exit 0;

__END__

    [download]
[reply]
[d/l]
      What, do you sit there and refesh this page all day?? Geez.. didn't even have time to get a cup of coffee..

      That's very cool.. thanks for you suggestion there and yes, I can do the network setup.

      webadept

      Every day someone is doing what someone else said is impossible.
[reply]
        Welcome to the wonders of the Newest Nodes page. :) With a minimum of about 4 - 5 users browsing the site at all times it is quite likely that you will get replies nearly instantly, at least if what you're asking about is not extraordinarily exotic.

        Makeshifts last the longest.

[reply]

Back to Seekers of Perl Wisdom