in reply to How to redefine a modules private function?

However it doesn't work. What am I doing wrong?

AnyEvent::DNS::DOMAIN_PORT is a constant function, which means it gets inlined, and within AnyEvent::DNS, the instances of DOMAIN_PORT are replaced by 53 by the compiler by the time the module is loaded and before control returns to your code. (See e.g. perl -MO=Deparse "`perldoc -l AnyEvent::DNS`")

At the moment, I don't see a way to modify the value of DOMAIN_PORT within AnyEvent::DNS itself from the outside, meaning you may have to resort to modifying the module's code. Constant Functions explains several ways to prevent inlining.

(There may be some Devel::* modules or some other trickery that could prevent inlining from the outside, but I'm currently unaware of any such modules, though I would be interested to learn about such things.)

Replies are listed 'Best First'.
Re^2: How to redefine a modules private function?
by cavac (Prior) on Mar 08, 2022 at 12:54 UTC

    I tried using source filters (Filter::Simple), but couldn't make it work, either. I usually don't use these, so i might have done it wrong.

    Frankly, i would consider the use of a constant function (or other type of constant) for a port number a major bug in AnyEvent::DNS. There are very few use cases that make sense to declare a number a constant in programming code. Pi might qualify, but in my opinion even physics constants shouldn't (because, hey, i might want to run a simulation to see what would happen if i change the planck constant.

    As for port numbers, if you want to run a network service as non-root (which is highly recommended, and pretty much a must while debugging software), the easiest way would be to change the port and set a fowarding rule on local the firewall. This is pretty much the default answer on most Linux forums. So coding a port number as constant in a library is a big no-no. Especially if it's a reserved port (lower than 1024).

    I haven't reported this as a bug to the AnyEvent maintainers, since i don't use AnyEvent. This should be done by sectokia, since they can provide more information to the maintainers.

    perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
      Frankly, i would consider the use of a constant function (or other type of constant) for a port number a major bug in AnyEvent::DNS. There are very few use cases that make sense to declare a number a constant in programming code.

      I assume you're not advocating for magic numbers - I might have said that a constant that isn't modifiable from outside the module is a problem.

      I tried using source filters (Filter::Simple), but couldn't make it work

      This was a fun project. I didn't exactly use source filters, but I did use PPI to munge the module before loading it, basically like a filtered use. Obviously this still has all the limitations of source filters and PPI!

      Deconstifier.pm

      package Deconstifier; use warnings; use strict; use parent 'PPI::Transform'; =head1 DESCRIPTION L<PPI::Transform> implementation that modifies a subset of L<constant functions|perlsub/"Constant Functions"> such that they are +no longer inlined. The subset of C<sub> definitions that are currently supported + is: sub FOO () { 42; } sub BAR () { "string"; } sub QUZ () { undef; } where the final semicolon is optional. =cut sub document { ( my $self = shift )->isa(__PACKAGE__) or return undef; ( my $doc = shift )->isa('PPI::Document') or return undef; my $subs = $doc->find(sub { if ( $_[1]->isa('PPI::Statement::Sub') && defined($_[1]->proto +type) && $_[1]->prototype eq "" ) { my $bl = $_[1]->block; if ( $bl && $bl->schildren==1 && $bl->schild(0)->isa('PPI: +:Statement') ) { my $st = $bl->schild(0); if ( $st->schildren==1 || $st->schildren==2 && $st->sc +hild(1)->isa('PPI::Token::Structure') && $st->schild(1)->content eq ' +;' ) { my $ch = $st->schild(0); if ( $ch->isa('PPI::Token::Number') || $ch->isa('P +PI::Token::Quote') || $ch->isa('PPI::Token::Word') && $ch->literal eq + 'undef' ) { return 1; } } } } return 0; }); return undef unless defined $subs; return 0 unless $subs; for my $s (@$subs) { #use PPI::Dumper; PPI::Dumper->new($s, whitespace=>0, comments +=>0)->print; # This first one only seems to work on Perl 5.8+, the second d +own to 5.6 and maybe/likely earlier (untested). # NOTE: This isn't really the right way to use PPI::Token::Wor +d, but since it's the only modification we're making it works fine. #$s->block->schild(0)->schild(0)->insert_before(PPI::Token::Wo +rd->new('return ')); $s->block->schild(0)->schild(0)->insert_after(PPI::Token::Word +->new(' if $]')); } return 0+@$subs; } 1;

      deconstify.t

      use warnings; use strict; use Test::More tests=>4; BEGIN { use_ok 'Deconstifier' } my $code = <<'END'; sub MAX_PKT() { 4096.0 } sub DOMAIN_PORT() { 53; } sub resolver (); sub _enc_qd() { (_enc_name $_->[0]) . pack "nn", ($_->[1] > 0 ? $_->[1] : $type_id {$_->[1]}), ($_->[3] > 0 ? $_->[2] : $class_id{$_->[2] || "in"}) } sub _enc_rr() { die "encoding of resource records is not supported"; } sub HELLO { "world" } sub WORLD () { "foo" } sub FOO () { $bar } sub BAR () { return 123 } sub BLAH () { undef; } END my $exp = <<'END'; sub MAX_PKT() { 4096.0 if $] } sub DOMAIN_PORT() { 53 if $]; } sub resolver (); sub _enc_qd() { (_enc_name $_->[0]) . pack "nn", ($_->[1] > 0 ? $_->[1] : $type_id {$_->[1]}), ($_->[3] > 0 ? $_->[2] : $class_id{$_->[2] || "in"}) } sub _enc_rr() { die "encoding of resource records is not supported"; } sub HELLO { "world" } sub WORLD () { "foo" if $] } sub FOO () { $bar } sub BAR () { return 123 } sub BLAH () { undef if $]; } END my $trans = new_ok 'Deconstifier'; ok $trans->apply(\$code), 'apply'; is $code, $exp, 'output is as expected';

      FilterLoad.pm

      package FilterLoad; use warnings; use strict; use List::Util qw/ pairs pairkeys /; use PPI; use Module::Load::Conditional qw/ check_install /; use Module::Runtime qw/ use_module module_notional_filename /; =head1 DESCRIPTION Loads modules after passing them through the L<PPI::Transform> filter( +s) given in the C<use> statement. For example: use FilterLoad 'AnyEvent::DNS' => 'Deconstifier', SomeModule => 'Deconstifier'; =cut sub import { my ($class, @defs) = @_; my (%mods, %filts); for ( pairs @defs ) { my ($mod, $filt) = @$_; $filts{$filt}++; my $modfn = module_notional_filename($mod); $mods{$modfn}{name} = $mod; push @{ $mods{$modfn}{filts} }, $filt; } use_module($_) for keys %filts; our $_in_inc_hook; unshift @INC, sub { my ($self, $modfn) = @_; return if $_in_inc_hook; return unless exists $mods{$modfn}; local $_in_inc_hook = 1; # check_install calls @INC hooks! my $info = check_install(module=>$mods{$modfn}{name}) or die "could not find $modfn"; my $doc = PPI::Document->new($info->{file}); $_->new->apply($doc) for @{ $mods{$modfn}{filts} }; return \$doc->serialize; }; use_module($_) for pairkeys @defs; } 1;

      test.pl

      use warnings; use strict; use FilterLoad Foo => 'Deconstifier', 'AnyEvent::DNS' => 'Deconstifier'; sub Foo::ONE () { 444 } sub Foo::TWO { 555 } sub Foo::THREE () { 666 } Foo::go(); print "One=", Foo::ONE, ", Two=", Foo::TWO, ", Three=", Foo::THREE, "\ +n"; # I've manually edited the module to include a function # sub foobar { print "DOMAIN_PORT=", DOMAIN_PORT, "\n" } AnyEvent::DNS::foobar(); sub AnyEvent::DNS::DOMAIN_PORT () { 4242 } AnyEvent::DNS::foobar();

      Using Foo.pm from my node here, and modifying AnyEvent::DNS as noted in the code above, the output is:

      Subroutine Foo::ONE redefined at test.pl line 9. Subroutine Foo::TWO redefined at test.pl line 10. Subroutine Foo::THREE redefined at test.pl line 11. Subroutine AnyEvent::DNS::DOMAIN_PORT redefined at test.pl line 19. One=444, Two=555, Three=666 One=444, Two=555, Three=666 DOMAIN_PORT=4242 DOMAIN_PORT=4242

        I assume you're not advocating for magic numbers - I might have said that a constant that isn't modifiable from outside the module is a problem.

        I agree. I meant "don't use constants, make it variables that can be changed from the outside".

        That said, i'm a lazy bum and i use some magic numbers in code i write.

        • I almost always use the magic numbers 1 and 0 for true and false. I'm just assuming that anyone reading perl code knows what that means.
        • When writing web stuff, i use just the HTTP status codes instead of their names. So i'd write 404 instead of instead of something like $HTTP_STATUS_CODES{'Not Found'}. Force of habit, i guess, because in my brain they're exactly the same thing.
        • When doing basic computer maths like bitshifting, i'm just assuming anyone reading the code understands that $x >> 8 means "shift by one byte to the right".
        • Same goes for $x & 0xff meaning "give me the last 8 bits"

        In my personal opinion, the worst offender in the whole Perl ecosystem when it comes to "magic numbers" is the default variable $_. It has no fixed value, and the perl interpreter pops it silently into your code whenever you forget to specify a variable.

        perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
Re^2: How to redefine a modules private function?
by syphilis (Archbishop) on Mar 08, 2022 at 10:34 UTC
    AnyEvent::DNS::DOMAIN_PORT is a constant function, which means it gets inlined, and within AnyEvent::DNS, the instances of DOMAIN_PORT are replaced by 53...

    I didn't know about this, and I don't want to hijack this thread, but I do have a tangential question.
    I'm wondering if there is essentially no difference between doing:
    sub DOMAIN_PORT() { 53 }
    and
    use constant DOMAIN_PORT => 53;
    Is there any difference at all ?

    Cheers,
    Rob
      I'm wondering if there is essentially no difference between doing: sub DOMAIN_PORT() { 53 } and use constant DOMAIN_PORT => 53; Is there any difference at all ?

      AFAIK not really, other than that constant does some checks on the constant names and that it keeps track of declared constants in %constant::declared - but they should both produce a constant function. (Even today you can still find the line *$full_name = sub () { $scalar }; in constant.pm.)

Re^2: How to redefine a modules private function?
by salva (Canon) on Mar 08, 2022 at 12:02 UTC
    There may be some Devel::* modules or some other trickery that could prevent inlining from the outside, but I'm currently unaware of any such modules, though I would be interested to learn about such things.

    A relatively simple solution would be to tie AnyEvent::DNS stash, so that the definition of DOMAIN_PORT could be replaced by a custom one:

    #!/usr/bin/perl use strict; use warnings; use feature 'say'; use Data::Dumper; use Tie::Hash; package Tie::Hash::Mine { BEGIN { our @ISA = qw(Tie::StdHash) }; sub STORE { warn "Setting AnyEvent::DNS::$_[1]"; $_[0]->SUPER::STORE($_[1], ($_[1] eq 'DOMAIN_PORT') ? sub () { 1053 } + : $_[2]) } }; BEGIN { tie %AnyEvent::DNS::, 'Tie::Hash::Mine'; } #use AnyEvent::DNS; <-- uncommenting this results in a segmentation fa +ult! BEGIN { package AnyEvent::DNS { sub ONE () { 1 } sub DOMAIN_PORT () { 53 } } } say Dumper \%AnyEvent::DNS::; say Dumper tied(%AnyEvent::DNS::);

    ... unfortunately, it seems that tieing a stash doesn't work at all :-(

      A very interesting hack attempt :-) I did briefly peek into the Perl source because I was curious if there was an obvious way to turn off the inlining (e.g. cv_const_sv), but I didn't see anything yet.

Re^2: How to redefine a modules private function?
by salva (Canon) on Mar 09, 2022 at 10:03 UTC
    Well, and now, something that actually works:
    #!/usr/bin/perl use strict; use warnings; use feature 'say'; use Path::Tiny; use File::Temp qw(tempfile); sub hotpatch { if ($_[1] eq 'AnyEvent/DNS.pm') { for my $inc (@INC) { next if ref $inc; my $fn = path($inc)->child($_[1]); if (open my $in, '<', $fn) { my ($out) = tempfile(UNLINK => 1); while (<$in>) { s/sub\s+DOMAIN_PORT\b/sub DOMAIN_PORT () { 1053 } +sub FORMER_DOMAIN_PORT/; print {$out} $_; } seek($out, 0, 0); return $out; } } warn "couldn't patch AnyEvent::DNS"; } return undef; } BEGIN { unshift @INC, \&hotpatch } use AnyEvent::DNS; BEGIN { @INC = grep not(ref and $_ eq \&hotpatch), @INC } say AnyEvent::DNS::DOMAIN_PORT(); say AnyEvent::DNS::FORMER_DOMAIN_PORT();

      You can avoid hitting the disk (even though that's great for debugging) by using an in-memory file in your @INC hook:

      open my $out, \my $buffer or die "Couldn't patch AnyEvent::DNS; your Perl do +es not support in-memory filehandles"; while (<$in>) { s/sub\s+DOMAIN_PORT\b/sub DOMAIN_PORT () { 1053 } +sub FORMER_DOMAIN_PORT/; $buffer .= $_; } return $out;
        Ok, here it is the full solution using an in-memory file:
        use strict; use warnings; use Path::Tiny; sub hotpatch { my ($self, $module) = @_; if ($module eq 'AnyEvent/DNS.pm') { for my $inc (@INC) { next if ref $inc; if (open my $in, '<', path($inc)->child($module)) { my $buffer = do { undef $/; <$in> }; $buffer =~ s/\bsub\s+DOMAIN_PORT\b/sub DOMAIN_PORT () +{ 1053 } sub FORMER_DOMAIN_PORT/; open my $out, '<', \$buffer or die "Couldn't patch AnyEvent::DNS; your Perl do +es not support in-memory filehandles"; return $out; } } warn "couldn't patch AnyEvent::DNS"; } return undef; } BEGIN { unshift @INC, \&hotpatch } use AnyEvent::DNS; BEGIN { @INC = grep not(ref and $_ eq \&hotpatch), @INC } say AnyEvent::DNS::DOMAIN_PORT(); say AnyEvent::DNS::FORMER_DOMAIN_PORT();
Re^2: How to redefine a modules private function?
by bliako (Abbot) on Mar 08, 2022 at 15:36 UTC

    Constant Functions says:

    Calls made using & are never inlined.

    I can't understand to what cases it applies to though.

    bw, bliako

      Calls made using & are never inlined. ... I can't understand to what cases it applies to though.

      In this case, this would require one to edit AnyEvent::DNS and change lines such as my $sa = AnyEvent::Socket::pack_sockaddr (DOMAIN_PORT, $server); to my $sa = AnyEvent::Socket::pack_sockaddr (&DOMAIN_PORT(), $server);.

      > I can't understand to what cases it applies to though.

      &foo is disabling any prototype checks on foo calls at compile time. But constant folding requires an empty prototype () to rule out any side effects at run-time.

      (Though I never tried playing around with other side effects like returning a closure var)

      Like HaukeX said, this doesn't help here, because you'd need to patch the source.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery