Stopping a package from infesting my namespace

anneli has asked for the wisdom of the Perl Monks concerning the following question:

Dear sisters and brothers,

I'm using Parse::Lex to generate lexers for a lexer/parser for Erlang source that I'm writing.

Fortunately or unfortunately, when you bootstrap a Lex object, it creates symbol table entries in the calling package for the tokens you've defined for the lexer. The code doing that is Parse/Token.pm, saying:

sub exportTo {
  my $self = shift;
  my $inpkg = $self->inpkg;
  unless (defined $inpkg) {
    $inpkg = caller;            # (caller(0))[0];       
    $self->inpkg($inpkg);
  }
  my $name = $self->name;
  no strict 'refs';
  if ($^W and defined ${"$inpkg" . "::" . "$name"}) {
    require Carp;
    Carp::carp "the '${inpkg}::$name' token is already defined";
  }
  ${"$inpkg" . "::" . "$name"} = $self;
  $self;
}
[download]

I'm trying to make my generated lexers at least vaguely re-entrant, so the lexer symbol table is being recreated each time you make a new lexer, meaning I'm getting these lovely "the 'XYZ::ABC' token is already defined" errors. It looks like there's no such thing as an anonymous package, so I can't enter one of those before generating the symbols.

Is there other way (possibly involving black magic) to get these tokens to go elsewhere, or otherwise stop this happening? (preferably without having to mess up Parse::Lex)

Yours in deep respect,

Anneli

Comment on Stopping a package from infesting my namespace Select or Download Code

Replies are listed 'Best First'.
Re: Stopping a package from infesting my namespace by armstd (Friar) on Oct 08, 2011 at 22:37 UTC
Well, how about "mess with" instead of "mess up"? Redefine package Parse::Token::exportTo with your own replacement. First make sure the original gets read in. Then just define it yourself. Hopefully Parse::Token doesn't do anything at compile time that requires it to be compiled in a specific order. `require Parse::Token; { # turn off warnings to avoid "Subroutine exportTo redefined" message no warnings; package Parse::Token; sub exportTo { ... } } package myPackage; ...` [download] I don't know enough (read: anything at all) about Perse::Lex to know how much messing you can get away with, but it sounds like you've already figured out most of that. It looks like Parse::Token::exportTo() only uses object data and nothing lexically scoped in the module, so at least that won't be a limiting factor. --Dave	[reply] [d/l]
Re^2: Stopping a package from infesting my namespace by anneli (Pilgrim) on Oct 08, 2011 at 23:37 UTC
Not bad! Thanks for your suggestion! I gave that a go; unfortunately it looks like Parse::Lex depends on those syms being there, and I hacked at it for quite some time without convincing it that they should be elsewhere. (I'd probably have to study it for some time longer, but the long and short of it is that I'd be monkeypatching a lot more than just exportTo(), so I'll leave it.) For now I'll just leave my module non-reentrant, until I work out another way, or maybe switch lexer generator. Thanks again! Anne	[reply]
Re^3: Stopping a package from infesting my namespace by armstd (Friar) on Oct 09, 2011 at 02:20 UTC
I guess my question then is are the reused symbols/tokens really a problem, or is it just the warnings? Do the tokens need to be scoped tighter than global to avoid conflict? If it's just the warnings, you can turn them off a variety of ways, like using 'use warnings;' (lexically scoped to just your own code) instead of 'perl -w' (global). If you do modify exportTo(), you can also just remove the Carp call. That 'if ($^W)' is just checking to see if warnings are enabled. --Dave	[reply]
Re^4: Stopping a package from infesting my namespace by anneli (Pilgrim) on Oct 09, 2011 at 03:34 UTC
Re: Stopping a package from infesting my namespace by GrandFather (Saint) on Oct 09, 2011 at 00:04 UTC
Can you use local to turn the trick or you? True laziness is hard work	[reply]
Re^2: Stopping a package from infesting my namespace by anneli (Pilgrim) on Oct 09, 2011 at 03:37 UTC
Possibly! I was thinking there may have been some related trick that I could use to stop this being such an issue, but it turns out I may have underestimated the amount of code-generation Parse::Lex was doing (and thus underestimated the dependence on these globals being present and so-named).	[reply]
Re: Stopping a package from infesting my namespace by Jenda (Abbot) on Oct 10, 2011 at 11:59 UTC
Seems you can tell the Parse::Token what package to create the symbols in by calling `$parse_token->inpkg($package_name)`. I thought the best way would be to specify the tokenClass option for Parse and subclass Parse::Token and do something in the constructor, but it seems there are problems with this ... the instance method tokenClass() of Parse::Lex (inherited from Parse::ALex) sets a "static" variable shared by all instances of any subclass of Parse::ALex so once you tell one instance to use a subclass for its tokens, all will. ... Seems to me the best you can do is to build and eval a string and hope for the best: `my $rand_pkg = 'AnneliTokens::T' . rand(); my $parser = eval "package $rand_pkg; Parse::Lex->new(\@token);";` [download] I do hope this will infest the rand package, not your package. Jenda Enoch was right! Enjoy the last years of Rome.	[reply] [d/l] [select]
Re^2: Stopping a package from infesting my namespace by anneli (Pilgrim) on Oct 10, 2011 at 21:31 UTC
Hey! Thanks for looking into this so much! I really appreciate it. Your findings with regards to tokenClass seem to be echoed throughout the design of Parse::Lex; it doesn't appear very friendly to the notion of more than one being instantiated (which is to say it can work, but you have to tiptoe around it!). OTOH, your eval trick looks pretty much perfect! I imagine it'll go into the new package this way (as it'll appear to be the callee), though it's a slightly hacky way of doing it. In the end, I just made my lexer non-reentrant, and it hides the unsightliness under the interface. But if I end up going about this another way, it'll likely be as you suggest! Thanks again! Anne	[reply]