in reply to Re^2: apache/perl caching problem
in thread apache/perl caching problem

Caching control in the HTTP headers controls what the clients (including intermediate proxy servers) do with the response and subsequent queries for the same URL. It doesn't matter whether the response came from a static HTML file or was produced by a script.

To be certain the responses are coming from your server, I would run a network sniffer on the server (e.g. wireshark) and observe the query and response.

After confirming the bad responses are coming from the server, I would investigate the server configuration. Given the unusual behavior, I would make no assumptions, so I would begin by determining what process is listening on the port that accepted the connection over which the request and bad response were exchanged. If this is an apache server process then the scope is narrowed, but perhaps there is some intervening software.

Is your script a standard CGI script? If it is, it will be read from disk and executed once for every request handled by the server. You might verify this by having your script log its start time, process ID, version and the full path and modification time of the file loaded. Log this every time your script handles a request. You can then correlate these logs with the requests to confirm that your script is running and producing the results you are seeing, and see exactly what is running each time. If you have plain CGI, you should see a different process ID each time and the version and modification times of your script should always be the latest. Your evidence suggests you will see something else. If you see the same process ID for several requests, then you should investigate what that process is, how it comes to be handling multiple CGI requests and how it is handling your script.

The mod_perl module was mentioned in a previous post. This and others can cause your script to be loaded and kept in memory, effectively becoming a subroutine that is executed over and over for each request, rather than running your script from disk for each request. This is caching of a different sort and seems a likely explanation for the behavior you have described. HTTP headers and meta tags in the produced HTML will not affect this sort of caching.

Replies are listed 'Best First'.
Re^4: apache/perl caching problem
by ksublondie (Friar) on May 01, 2010 at 16:43 UTC
    It looks like mod_perl is installed...When reloading apache, I see: 
    "Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 PHP/5.2.6-1+lenny8 with Suhosin
+-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0 confi
+gured -- resuming normal operations"

    [download]
    in the logs.

    Of course I know nothing about mod_perl and very little about how the server is configured. I've inherited the server from a previous co-worker.

    I'm going to try to do a little research on mod_perl, but in the meantime, is there anything I should look for if this is the problem? i.e. configuration?

[reply]
[d/l]

      Mod_perl improves performance by loading the perl interpreter and your program once and then keeping these in memory and using them to respond to several requests. Configuration parameters determine how many requests each process or thread handles.

      If your server has a pool of processes/threads handling requests, each started at a different time, then each might have a different version of your program loaded. When a new request arrives at the server, it will be passed to one of the available processes/threads. If the server is busy, which process/thread handles the next request will be quite random. Thus, it will sometimes be handled by a process/thread running a recent version of your program and sometimes by one running an old version.

      If you stop the server, all processes/threads will be stopped. When you restart it, all new processes/threads will load the then current version of your program. Thus a full shut down and restart of the apache server should solve your problem.

      If your users can't tolerate the service disruption, then you can do a "graceful" restart. This allows current processes/threads to finish handling their current requests but then they stop and new processes/threads are started and these will load the current version of your program.

[reply]
        I restarted the server and it doesn't look like it solved the problem. :-( I'm assuming then, that this is not related to mod_perl. Correct?

        "...Is your script a standard CGI script?..."

        Umm, as opposed to...??? Yes, I'm using CGI if that was your question. I'm sorry, I'm afraid you're a little over my head here. Everything I've learned about perl was on my own and doesn't extend much farther than coding and installing modules.

        "...You might verify this by having your script log its start time, process ID, version and the full path and modification time of the file loaded. Log this every time your script handles a request..."

        Is this different than the access logs? I was able to find an apache .pid file, but that simply contains the number "3067" and isn't changing.

        I'm checking out CGI::Log and CGI::LogCarp. Is something like that what you were referring to? Sorry if this is a stupid question: if I modify my code to include logging and I have a problem with my code caching, how would I be able to log my old script that doesn't log but is still in the cache?

        I don't know if this is helpful information, but I've been working on this site for well over a year. The problem started slowly at first. Gradually over time, this has been increasing. Now, I'm getting more of my old script versions than of the current versions. This site is the intranet and internal home page of my company of approximately 180 employees, so it gets a lot of hits throughout the day.

[reply]

In Section Seekers of Perl Wisdom