my $request = new HTTP::Request GET, $url, "Pragma: no-cache";
added commas.
s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s
|-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,,
$|=1,select$,,$,,$,,1e-1;print;redo}
| [reply] [d/l] |
Another question related to this... does HTTP::Request cache any information locally like a browser might? | [reply] |
HTTP::Request creates the request structure it does not fetch web pages. That function is provided by LWP::UserAgent. In your code you call the constructor for these two classes inside you subroutine, so the instance of the class is local to the subroutine and goes out of scope when you exit the subroutine. This should take care of any data persistance problem. So your answer is no.
Note that memory for the instance is not released until the program exits so that memory is reserved everytime your subroutine runs but is not released. This is a memory leak and could be causing problems depending on how many times your subroutine is called.
Note: I corrected a syntax error in my other post.
s/net/not/
s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s
|-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,,
$|=1,select$,,$,,$,,1e-1;print;redo}
| [reply] |
Hmmmm... yeah, this problem is driving me crazy. Thanks for the responses so far. I've tried sending the client header request with a no cache setting... I've also tried changing the document age to something very high so it will always be considered "not fresh". Something out there is caching this information and giving bad data... what's strange is the URL I'm passing is different each time (dynamically generated by filling in parameters passed in from the URL). If I take the same exact URL and drop it into my browser I get the correct http response every single time. But, using LWP::UserAgent it's bad bad bad. What is strange too is every time I make a request the responding app creates a cookie with a session ID... I can see the cookie and it shows up and it's different everytime... so I assume the request is actually making it to the server. I see this problem with a proxy server and without.... BUT, if I move from one proxy to another every few hours I'll get one good response... then all repsonses after that are old and incorrect. The cache munchkins are toying with me! Arg!
Btw, in this particular situation the subroutine is only being called once... then the program exits. But, how would I get rid of this leak? Thanks...
| [reply] |
Hmm... well, making forward progress... kind of. The http header response I am getting back has this info available via $response->fresh_until etc:
Fresh Until: 1/16/2008 16:16:15
Current Age: 0
Freshness Lifetime: 9/26/1972 17:00:00
Is Fresh: 1
I've added this to the request:
$request->header("Pragma" => "no-cache");
$request->header("Cache-control" => "no-cache");
I even tried:
$request->header("Cache-control" => "max-age=-1");
No luck... any http guru's out there? How can I control what the age is and ensure that my request rejects this doc that for some reason has a riculously large lifretime? I'm assuming the issues is whith my http header since my browser sends back the correct data every time... hmmmpf.
| [reply] [d/l] |
I fixed up the object allocation and ran this on a timeserver that updates once per second. Had no problems.
my $ua = LWP::UserAgent->new(
agent => 'Mozilla/4.0 (compatible; MSIE 6.0; Win32)');
while () {
my $string=get_response( $url, $ua);
$string=~/\d+:\d+:\d+/;
print $&,"\n";
};
sub get_response {
my ( $url, $ua ) = @_;
my $request = HTTP::Request->new( GET => $url );
my $response=$ua->request($request);
return $response->as_string;
};
Note that there is no need to explicitly process 301 redirects. LWP::UserAgent handles these automaticly. From LWP::UserAgent
"The request() method will process redirects and authentication
responses transparently."
s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s
|-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,,
$|=1,select$,,$,,$,,1e-1;print;redo}
| [reply] [d/l] |