Re^5: Unfinished custom proxy

Hi again. I've managed to start a proxy with your help. And now it even serves some requests ( it's some log entries):

[Sun Jan  8 10:38:53 2012] (1554) SOCKET: New connection from 127.0.0.
+1:49293
[Sun Jan  8 10:38:53 2012] (897) PROCESS: Forked child process 1554
[Sun Jan  8 10:39:01 2012] (1554) REQUEST: GET http://www.google.com/s
+orry/?continue=http://www.google.com/search%3Fq%3Dcar+rent
[Sun Jan  8 10:39:01 2012] (1554) REQUEST: Via: 0.9 domU-00-05-34-21-B
+5-AC.compute-1.internal (HTTP::Proxy/0.25)
[Sun Jan  8 10:39:01 2012] (1554) REQUEST: X-Forwarded-For: 127.0.0.1
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: 503 Service Unavailable
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Cache-Control: no-cache, m
+ust-revalidate
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Date: Sun, 08 Jan 2012 15:
+39:01 GMT
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Via: 1.1 domU-00-05-34-21-
+B5-AC.compute-1.internal (HTTP::Proxy/0.25)
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Server: HTTP server (unkno
+wn)
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Content-Length: 2639
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Content-Type: text/html; c
+harset=UTF-8
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Expires: Mon, 01 Jan 1990 
+00:00:00 GMT
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: Title: http://www.google.c
+o.uk/search?q=car rent
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: X-Cache: MISS from domU-00
+-05-34-21-B5-AC.compute-1.internal
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: X-Cache-Lookup: MISS from 
+domU-00-05-34-21-B5-AC.compute-1.internal:3128
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: X-Frame-Options: SAMEORIGI
+N
[Sun Jan  8 10:39:01 2012] (1554) RESPONSE: X-XSS-Protection: 1; mode=
+block
[Sun Jan  8 10:39:01 2012] (1554) SOCKET: Connection closed by the pro
+xy
[Sun Jan  8 10:39:01 2012] (1554) PROCESS: Served 1 requests
[Sun Jan  8 10:39:02 2012] (897) PROCESS: Reaped child process 1554
[Sun Jan  8 10:39:02 2012] (897) PROCESS: 0 remaining kids:
[download]

But the link is not being transformed.

Comment on Re^5: Unfinished custom proxy Download Code

Replies are listed 'Best First'.
Re^6: Unfinished custom proxy by Eliya (Vicar) on Jan 08, 2012 at 19:57 UTC
I guess you need to unescape the URI in `$new_location`, i.e. `use URI::Escape; my $location = "http://www.google.com/sorry/?continue=http://www.googl +e.com/search%3Fq%3Dperlmonks"; my $new_location = $location; $new_location =~ s/.\/sorry\/\?continue=(.)/$1/; $new_location = uri_unescape($new_location); print " $location\n=> $new_location\n"; __END__ http://www.google.com/sorry/?continue=http://www.google.com/search% +3Fq%3Dperlmonks => http://www.google.com/search?q=perlmonks` [download] That said, I don't get any captchas or sorries from Google (is this still your squid's sorry thing?), so I couldn't test it as is... but when I submit an incorrect search, e.g. `http://www.google.com/search?=perlmonks` [download] (i.e. missing `q`, as in your OP), Google redirects me to `http://www.google.com/webhp`, so I used that response to test, and when I rewrite the Location to `if ($location =~ m\|/webhp$\|) { my $new_location = "http://www.google.com/search?q=perlmonks"; $headers->header( Location => $new_location ); }` [download] things work fine for me, while with the escaped `http://www.google.com/search%3Fq%3Dperlmonks` I do get a 404 Google custom error page.	[reply] [d/l] [select]
Re^7: Unfinished custom proxy by kazak (Beadle) on Jan 10, 2012 at 10:22 UTC
Hi. How do you think will it be difficult to implement a true repetition of a "sorry link"? As for now script, made with your help and help of other honorable monks, can: - Detect, this link - Rewrite "location" header from wrong one to right one. - Fully log its activity. So as far as I can see in order to repeat this link, we need to save a session ID of the client, request this content again and, retrieve this content to a client with appropriate session ID, is this right, how do you think ?	[reply]