unwanted cache when using LWP to get content of a page

gooyava has asked for the wisdom of the Perl Monks concerning the following question:

I'm using a perl script via my shell console (SSH) to get contents of a file on a different server. the first time I run this script I get the right content of the file, but I change the content of the file and than try to run the code again I get the content as before - the first time.

my $url='http://mysite.com/test.html';
my $proxy_url='http://myproxy.com';

my $browser = LWP::UserAgent->new;
$browser->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)');
$browser->timeout(10);
$browser->proxy(['http', 'ftp'], $proxy_url);

my $request = HTTP::Request->new(GET => $url)|| print($!);
my $response = $browser->request($request)|| print($!);

if ($response->is_success)
{
 my $content= $response->content;
 return $content;
}
else
{
 my $content= $response->status_line();
 return $content;
}
[download]

Ignore the actual url's in $url and $proxy_url - they are fictinial for the porpose of this post. no matter how many times I change the content of "test.html" $content keeps giving me the same result it gives me the first time.

is this some kind of server cache ? proxy cache ? what ? where to start ?

Comment on unwanted cache when using LWP to get content of a page Download Code

Replies are listed 'Best First'.
Re: unwanted cache when using LWP to get content of a page by JavaFan (Canon) on Nov 10, 2008 at 12:21 UTC
Considering that LWP doesn't do caching, it looks to me that either the server, or the proxy is caching. And I'd look at the proxy first. Note that it may not be the proxy's fault if the server suggest that the data doesn't expire, or hasn't expired yet when the data does. I'd start looking at the proxy and server log files.	[reply]
Re: unwanted cache when using LWP to get content of a page by almut (Canon) on Nov 10, 2008 at 13:36 UTC
If all else fails, caching problems can sometimes be worked around by appending some unique dummy parameter to the URL (e.g. use current time in secs, or some counter): `my $url='http://mysite.com/test.html?whatever=1226323589';` [download] With a bit of luck, this should trick the proxy into believing you're requesting a dynamic resource (which it normally wouldn't cache), or - if that doesn't suffice - the different parameter value would pretend you're requesting a different resource every time...	[reply] [d/l]
Re: unwanted cache when using LWP to get content of a page by massa (Hermit) on Nov 10, 2008 at 13:38 UTC
Hi! You should try to use `<code>` tags instead of whatever you used, because your post is scrambling the formatting of the whole page. Try putting `$request->header( 'Cache-Control' => 'no-store' ); $request->header( Pragma => 'no-cache' );` [download] before `my $response = $browser->request #...` [download] and that should avoid your proxy from keeping the cache. []s, HTH, Massa (κς,πμ,πλ)	[reply] [d/l] [select]