njaph has asked for the wisdom of the Perl Monks concerning the following question:

Hi All,

I'm just starting to test a migration of a mod_perl app from a debian woody (perl 5.6.1, mod_perl 1.26-3.0) to debian sarge (perl 5.8.4, mod_perl 1.29.0.3) and I'm running into a problem with a package level my() variable accessed in package level subs. See below for quick example. This is code I haven't had to touch for 18 months :)

I've had to change the my() to our() but I don't understand why - it feels like I'm missing something very obvious :)

I've had a look at all the perl*delta pod and the changelog for mod_perl and didn't see anything obvious so I'm asking here - your wisdom is appreciated!

The following is part of a simple user details cache module used in an apache auth handler (abbreviated so don't try too hard to syntax check it):

package My::Apache::UserCache; use strict; use My::User; require Exporter; our @ISA = qw(Exporter); our @EXPORT_OK = qw(get_pass get_groups); use constant CACHE_TIMEOUT => 60; # seconds my %cache = (); sub fill_cache { my $user = shift; my $now = time; my ($password,@groups) = My::User->get_pw_groups( $user ); # hits DB ... other checks ... $cache{$user} = { timestamp => $now, password => $password, groups => [@groups] }; } sub get_pass { my $user = shift; my $now = time; return $cache{$user}->{password} if $exists_and_hasnt_timed_out_etc; return undef unless fill_cache( $user ); $cache{$user}->{password}; # get direct from cache } sub get_groups { # similar to above }
Example of code that calls it in Authen/Authz handlers:
package My::Apache::UserAuthen; use strict; use My::Apache::UserCache qw(get_pass); ... sub handler { ... setup; get apache basic auth credentials; etc ... my $reason = authenticate( $user, $user_pw ); ... log etc on failure.. return OK; } sub authenticate { my ($user, $user_pw) = @_; ... my $db_pass = get_pass( $user ); ... return messages based on match/mismatch/etc ... return ''; }

Offline using the cache module directly things work fine either way, things work fine in a single page request under 5.6.1/1.26 but the cache variable never keeps values when using my() with a single page request under 5.8.4/1.29. Like I said, changing to it to our() works but why didn't it before?

I use this method for package variables a lot so I'd really like not to have to update and test tens of modules along the way. What, if anything, should I be using instead?

I look forward to hearing peoples suggestions, fixes and opinions - thanks all.

Cheers,

Andrew

Replies are listed 'Best First'.
Re: package scoped my() variable in module under mod_perl 5.6 vs 5.8
by perrin (Chancellor) on Sep 20, 2005 at 04:19 UTC
    I assume you're talking about %cache? I don't think there's anything that changed in those versions which would affect this. How are you measuring that it doesn't retain its values?
Re: package scoped my() variable in module under mod_perl 5.6 vs 5.8
by shenme (Priest) on Sep 20, 2005 at 04:30 UTC
    I'm wondering if having problems with "hits DB" might not end up looking like your cache isn't being populated, which it wouldn't be. :) I guess I'd like you to check that %cache has values at the end of fill_cache() before you worry that the values might not stick around. (I'm having trouble believing just changing the my to our really fixes it.) Can you show more code in fill_cache()?
      As I mentioned - this code has been running fine in a production server for 18 months or more - the previous code definitely worked.

      I'm quite willing to believe there is something else in the environment that is getting in the way - I was just very surprised (and, happily, judging by ppls reactions so far, so are they) that the my() variable wasn't filled in.

      And, yes, I've tested that fill_cache gets values and puts them into %cache. The problem is that the values don't hang around once fill_cache() returns. I don't know why :-)

      Output from full code below is here. First set of statements is with my(), second with our(). Full apache restart just before each request.

      A normal request involves hitting the cache for password then again for a list of allowed groups (in two separate Auth* handlers) and checking against the Apache require group directive. You can see that the group lookup isn't even mentioned on the successful run as it is already cached.

      [18981] get_pass: not in cache - retrieving [18981] fill_cache: setting cache [andrewo] pw:[andrewo] groups:[enabl +ed] [18981] get_pass: password now: andrewo [18981] get_groups: not in cache - retrieving [18981] fill_cache: setting cache [andrewo] pw:[andrewo] groups:[enabl +ed] [18981] get_groups: groups now: user undef! [Tue Sep 20 17:15:51 2005] [error] Can't use an undefined value as an +ARRAY reference at /usr/local/share/perl/5.8.4/Oriel/Apache/UserCache.pm line 56.\n [19054] get_pass: not in cache - retrieving [19054] fill_cache: setting cache [andrewo] pw:[andrewo] groups:[enabl +ed] [19054] get_pass: password now: andrewo
      And the full UserCache module (with added warns):
      package My::Apache::UserCache; use strict; use Apache::Constants qw(:common); use My::User; require Exporter; our @ISA = qw(Exporter); our @EXPORT_OK = qw(get_pass get_groups); use constant CACHE_TIMEOUT => 60; # seconds # # username => { timestamp => unixtime, # password => blah, # groups => [groups], }, # our %cache = (); sub fill_cache { my $user = shift; my $now = time; my ($password, @groups) = Oriel::User->get_pw_groups($user ); # enforce at least 1 character passwords return 0 unless defined $password && length($password); warn "[$$] fill_cache: setting cache [$user] pw:[$password] groups:[ +".join(',',@groups)."]\n"; $cache{$user} = { timestamp => $now, password => $password, groups => [@groups] }; } sub get_pass { my $user = shift; my $now = time; return $cache{$user}->{password} if ( exists $cache{$user} && $cache{$user}->{timestamp} && ($now - $cache{$user}->{timestamp}) < CACHE_TIMEOUT ); warn "[$$] get_pass: not in cache - retrieving\n"; return undef unless fill_cache( $user ); warn "[$$] get_pass: password now: ".(defined cache{$user} ? $cache{ +$user}->{password} : 'user undef!')."\n"; $cache{$user}->{password}; } sub get_groups { my $user = shift; my $now = time; return @{$cache{$user}->{groups}} if ( exists $cache{$user} && $cache{$user}->{timestamp} && ($now - $cache{$user}->{timestamp}) < CACHE_TIMEOUT ); warn "[$$] get_groups: not in cache - retrieving\n"; return () unless fill_cache( $user ); warn "[$$] get_groups: groups now: ".(defined $cache{$user} ? @{$cac +he{$user}->{groups}} : 'user undef!')."\n"; @{$cache{$user}->{groups}}; } 1;

      Its weird. Any ideas?

Re: package scoped my() variable in module under mod_perl 5.6 vs 5.8
by njaph (Initiate) on Sep 21, 2005 at 01:12 UTC

    OK, I've narrowed this down to something funky in Exporter. Removing all traces of Exporter from the UserCache module and calling subs by fully qualified name in the Apache Auth* handlers has completely fixed it and life has returned to normal - ie my() works as expected.

    I'm going to have much more of a poke through the mod_perl mailing lists now and see if this is a manifestation of the accidental closure issue.

    Thanks for helping me bounce this around. If anyone has any ideas why this behaviour happens when using Exporter as above then please please let me know :)

    Cheers!

      It would not be an accidental closure, since what you have here is an intentional closure. If using Exporter breaks the closure, the right thing to do is to check the Perl changelog for bits about that and if you don't see anything, make a small example and post it first on the mod_perl list and then on perl5-porters.
Re: package scoped my() variable in module under mod_perl 5.6 vs 5.8
by Errto (Vicar) on Sep 20, 2005 at 15:52 UTC
    If the issue really is a difference between my() and our(), then the only thing I can think of is to check your Apache logs for when the server is first starting up and look for warning messages that look like "variable %cache will not stay shared at...." (you are using warnings, right?) This is something you should not normally see in a handler module, unless for some bizarre reason this module were being processed by Apache::Registry. I confess to not being a mod_perl expert, but I don't see how that could actually happen. Nonetheless, I would check your Apache configuration files to make sure this is not the case, and also check to make sure that you're not useing or requireing this My::Apache::UserCache module from within a subroutine in some other module.