in reply to apache/perl caching problem

but I could receive any version of my perl at any time.

I guess you mean that at your browser you receive the output produced when your server runs your Perl scripts. If I have guessed wrong then what follows probably isn't relevant.

If you sometimes receive old content after having received new content then it isn't a simple caching problem.

Do you have a pool of caching proxy servers between your browser and server? If so, each proxy server might have cached a different version of response and, depending on which server the load balancer sends your latest request to, you might get various ages of responses somewhat randomly. Controlling expiry will solve this problem if the proxy servers are well behaved. In development I would bypass all proxy servers and query the source server directly, if at all possible, until I was sure the server was stable and correct, then test through the proxy servers to ensure that caching wasn't causing a problem.

If there are proxy servers between browser and origin server, can you inspect their logs to find out what requests they receive and what they do to respond to them?

this is a server problem not a client problem, right?

When you receive old content at your browser, does your server access log show that the request made it all the way to the server? Your content may be delivered in parts through several requests: a main document and many linked documents (images, css, javascript, etc.). If content is not being served from a cache then you should see a record of each part being accessed in your server's access log.

Setting the cache expiry on the document produced by your Perl script will not affect caching of other linked documents.

If you are going to control caching, it would be better to put the controls in the HTTP headers rather than in the HTML. You can't add meta tags to images, as far as I know, and I suspect not in your javascript either (though I could be wrong about the latter). There is a nice tutorial at http://www.mnot.net/cache_docs/ that explains some of the options and issues.

You may find the apache mod_expires module helpful for adding HTTP headers related to caching: http://httpd.apache.org/docs/2.0/mod/mod_expires.html

Replies are listed 'Best First'.
Re^2: apache/perl caching problem
by ksublondie (Friar) on Apr 23, 2010 at 18:27 UTC
    "...you sometimes receive old content after having received new content..."

    That is EXACTLY what I'm getting...and it could be any version in between! I'm not sure on the proxy servers, but I doubt it. As far as I know this is a basic intranet server. I need to hear back from the guy that does the network architecture to know for sure...

    I guess I should have been clearer on what I am getting. When I modify my HTML templates, the output is immediately correct. When I modify my perl script, I have to refresh several times to maybe see the change. ...and once I do see the change, I may get the older version at any time. The portion that is generated with the perl is seemingly random. Some crazy examples:

    • Now-fixed errors getting generated & triggering a separate error page (in the perl) to display instead of the correct output page
    • Email generation: either triggered by the example above and/or containing different content than the current version
    • SQL queries & output may be different -- including now-fixed SQL errors still being generated, ex. incorrect server connection variables, query typos, etc
    • Different variable values presented to the templates
    I suppose the last 2 examples could be a previously cached local version, but to me the email screams "server problem". Frustrating, to say the least.

    I haven't checked the access logs, but when I receive an error, it's in the error logs and of course the email error is generated. I can check the access log, of course it may take me a while to PURPOSELY recreate the problem!

    ...Yes, I've tried the ctrl+f5, clearing my cache multiple times, changing browser settings, blah blah blah, and anything else that would point to a local caching problem... I can also recall having end users test it for me that have NEVER been to the page, however they may get a previous version.

    I just scanned those links you referred and haven't read them in depth yet, but wouldn't controlling the caching in the HTTP headers only effect the HTML and not the perl behind the scenes?

[reply]

      Caching control in the HTTP headers controls what the clients (including intermediate proxy servers) do with the response and subsequent queries for the same URL. It doesn't matter whether the response came from a static HTML file or was produced by a script.

      To be certain the responses are coming from your server, I would run a network sniffer on the server (e.g. wireshark) and observe the query and response.

      After confirming the bad responses are coming from the server, I would investigate the server configuration. Given the unusual behavior, I would make no assumptions, so I would begin by determining what process is listening on the port that accepted the connection over which the request and bad response were exchanged. If this is an apache server process then the scope is narrowed, but perhaps there is some intervening software.

      Is your script a standard CGI script? If it is, it will be read from disk and executed once for every request handled by the server. You might verify this by having your script log its start time, process ID, version and the full path and modification time of the file loaded. Log this every time your script handles a request. You can then correlate these logs with the requests to confirm that your script is running and producing the results you are seeing, and see exactly what is running each time. If you have plain CGI, you should see a different process ID each time and the version and modification times of your script should always be the latest. Your evidence suggests you will see something else. If you see the same process ID for several requests, then you should investigate what that process is, how it comes to be handling multiple CGI requests and how it is handling your script.

      The mod_perl module was mentioned in a previous post. This and others can cause your script to be loaded and kept in memory, effectively becoming a subroutine that is executed over and over for each request, rather than running your script from disk for each request. This is caching of a different sort and seems a likely explanation for the behavior you have described. HTTP headers and meta tags in the produced HTML will not affect this sort of caching.

[reply]
        It looks like mod_perl is installed...When reloading apache, I see: 
        "Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 PHP/5.2.6-1+lenny8 with Suhosin
+-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0 confi
+gured -- resuming normal operations"

        [download]
        in the logs.

        Of course I know nothing about mod_perl and very little about how the server is configured. I've inherited the server from a previous co-worker.

        I'm going to try to do a little research on mod_perl, but in the meantime, is there anything I should look for if this is the problem? i.e. configuration?

[reply]
[d/l]
      mod_perl?
[reply]

In Section Seekers of Perl Wisdom