With a lot of debug information :

use warnings;
use strict;

sub A {

    print "\n\nentering, \$_[0] is $_[0]\n";

    print "entering, \$_[1] is $_[1]\n";
    if ($_[1] == 0) {print "1st -- returning 0\n"; return 0  ;}
    if ($_[0] == 0) {print "2nd -- returning 2*\$_[1] ($_[1])\n"; retu
+rn 2*$_[1] }
    if ($_[1] == 1) {print "3rd -- returning 2\n"; return 2 };
    print "not returning : ";
    # figure A(m-1, A(m, n-1)) using in-place parameter list to save s
+tack
    --$_[1];
    print "\$_[1] is $_[1] ";
    
    $_[1]= &A;

    print "\$_[1] is $_[1] ";
    --$_[0];
    print "\$_[1] is $_[1] ";
    return &A;
}

print A ($ARGV[0], $ARGV[1]);
[download]

gives :

C:\>trash 3 3


entering, $_[0] is 3
entering, $_[1] is 3
not returning : $_[1] is 2

entering, $_[0] is 3
entering, $_[1] is 2
not returning : $_[1] is 1

entering, $_[0] is 3
entering, $_[1] is 1
3rd -- returning 2
$_[1] is 2 $_[1] is 2

entering, $_[0] is 2
entering, $_[1] is 2
not returning : $_[1] is 1

entering, $_[0] is 2
entering, $_[1] is 1
3rd -- returning 2
$_[1] is 2 $_[1] is 2

entering, $_[0] is 1
entering, $_[1] is 2
not returning : $_[1] is 1

entering, $_[0] is 1
entering, $_[1] is 1
3rd -- returning 2
$_[1] is 2 $_[1] is 2

entering, $_[0] is 0
entering, $_[1] is 2
2nd -- returning 2*$_[1] (
$_[1] is 4 $_[1] is 4

entering, $_[0] is -1
entering, $_[1] is 4
not returning : $_[1] is 3

entering, $_[0] is -1
entering, $_[1] is 3
not returning : $_[1] is 2

entering, $_[0] is -1
entering, $_[1] is 2
not returning : $_[1] is 1

entering, $_[0] is -1
entering, $_[1] is 1
3rd -- returning 2
$_[1] is 2 $_[1] is 2

entering, $_[0] is -2
entering, $_[1] is 2
not returning : $_[1] is 1
[download]

so, looking at the issue in a purely symptomatic sense, the infinite recursion can be stopped by changing line 11 to :

    if ($_[0] <= 0) {print "2nd -- returning 2*\$_[1] ($_[1])\n"; retu
+rn 2*$_[1] }
[download]

, but I don't know how this affects what this sub does, since it only seems to produce 2 ** ARGV[1].

what gives? what is this code really about?

update AHA!
you appear to be implementing the algorithm incorrectly, if I may be so bold :-)