This thought came out of a question I've been puzzling about in my tiny-brained way, and which I mentioned in the CB yesterday. As well as raising my present topic I'm glad to have an excuse to put down the solution to the problem I discussed, just so anyone who was kind enough to offer advice can see that it didn't fall on deaf ears. Thanks again, sibling monks.

I'd also like to crave your indulgence for any foolishness I may commit in the following, as these questions of scope and naming are new to me and I find them quite difficult.

So, my thought. I think it might be nice if there was a way to make a subroutine behave, in regard to naming scopes, exactly the same as if, instead of calling the module, you had cut and pasted the lines of code into your script in place of the subroutine call.

Maybe there is a way to do this. If so, I'd love to know of it. Maybe you think it wd be a dangerous thing. But, whilst being open to persuasion, I don't agree: the variables are still contained within a lexical scope, only it's the lexical scope of the surrounding block. So if the subroutine creates or alters variables in the scope in which it's called, this is only a problem to the extent that one has been careless with one's variables and scoping anyway.

Set against this danger, is the advantage of (A) being able to give the subroutine access to all the variables existing in the calling scope without and (B) being able to pass back not just a return value but a variable (or several vars) which can be manipulated.

On the other hand, if you just don't see why I would be worried about this in the first place, follow the readmore tag...

In the set of scripts I'm working on now I often have to interrogate a database to get three records from three different tables, in the form of three hashrefs, which I then do a wide range of different operations on. The main place for code re-use here is the getting of the data from the db, not what I do with it. Now the problem is, my subroutine needs to return a hashref which I can then use freely after the subroutine is done. One way I tried to do it was:

First I have a module, let's call it booker.pm, which looks like this:
package booker; require Exporter; our @ISA = ("Exporter"); sub Get_Booking_Course_Event { my $BookingID = shift; my $dbh = DBI->connect("DBI:mysql:database=foo", "bar", "baz") or +die $DBI::errstr; my $sth; $sth = $dbh->prepare("SELECT * FROM booking WHERE BookingID = $Boo +kingID") or die $dbh->errstr; $sth->execute(); $ref_booking = $sth->fetchrow_hashref; $sth = $dbh->prepare("SELECT * FROM event WHERE EventID = $ref_boo +king->{'EventID'}") or die $dbh->errstr; $sth->execute(); $ref_event = $sth->fetchrow_hashref; $sth = $dbh->prepare("SELECT * FROM course WHERE CourseID = $ref_b +ooking->{'CourseID'}") or die $dbh->errstr; $sth->execute(); $ref_course = $sth->fetchrow_hashref; } our @EXPORT = qw/Get_Booking_Course_Event/; 1;
And use booking; in each of the scripts.

My first attempt at calling this subroutine was
my $ref_booking; my $ref_event; my $ref_course; &Get_Booking_Course_Event($ref->{'BookingID'}); # the arg in this sub cd be one of a number of # things depending on where in the script we are do_stuff_with($ref_booking, $ref_course, $ref_event);
Now, the main problem with that is it doesn't work. Because the three variables I declare before calling the subroutine don't get into the namespace of the subroutine... let alone out of it again with something useful in them.

My next try was to call it without declaring any variables and then use the variables from the module's namespace, with fully-qualified variable names:
&Get_Booking_Course_Event($ref->{'BookingID'}); # the arg in this sub cd be one of a number of # things depending on where in the script we are do_stuff_with($booker::ref_booking, $booker::ref_course, $booker:: +ref_event);
That seems to work. But I still don't totally like it. For one thing, I quite often call this sub more than once in the same script. And even though I have error checking, I'm slightly worried that something might go wrong later on, and then the second time I do my operations with $booker::ref_booking etc, I'm still working on the first ones.

What I'd really like is to be able to localise the variables that the subroutine returns. Or even better, have them returned ready-localised. Which is what finally led me to the desire to be able to make my subroutine be in the same name space as the calling scope. Then I'd declare $ref_booking and the others in the subroutine. And I also wouldn't have to keep creating and destroying my database handle, because I could use one I created earlier, which is in scope at the point where I call the sub.

So that's what I'd like to be able to do. And I don't think I can. So... tell me I can; or tell me I'm crazy to want to.


§ George Sherston

Replies are listed 'Best First'.
Re: subroutines and namespace
by dragonchild (Archbishop) on Nov 02, 2001 at 18:33 UTC
    Maybe I'm missing something extremely simple, but why don't you just end your subroutine with
    return ($ref_booking, $ref_course, $ref_event); }
    and then, when you call the subroutine, call it like:
    my ($ref_booking, $ref_course, $ref_event) = GetStuffFromDB($my, $parameters, $here);
    Or, is there some reason you didn't mention that would prevent this from happening?

    ------
    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

      Yes, that'd do it. Very neat - possibly nicer than the solution I went with, of using fully specified package variables from my module. Only disadvantage is a few more characters of code... which are marginal even if I'm intent on Higher Laziness.

      My main point was a general one, however, that sometimes one wants to have subroutines with the same scope as the point from which they're called. Just as often one does *not* want this. I wondered whether I was alone in thinking this might be a useful thing to be able to do - in a lot of different situations.

      § George Sherston
        Actually, you would never want to care where the subroutine's scope is. And, the reason why is that it would violate encapsulation.

        The way to think about a subroutine is that it's a self-contained unit of activity. It receives inputs, does stuff, and gives outputs. It should NEVER depend on the caller's scope. It should NEVER depend on anything but that which is within its own personal scope (which should be inviolate and untouchable by caller).

        Every single legitimate reason you would want to do what you suggest is negated by a language feature or design change that will make your code more maintainable and easier to work with.

        ------
        We are the carpenters and bricklayers of the Information Age.

        Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re: subroutines and namespace
by Masem (Monsignor) on Nov 02, 2001 at 17:03 UTC
    Maybe not the best suggestion, but consider it a seed for possibly other ideas:

    Why not have an optional hashref that you pass in, such that if it is defined, you store the results there, otherwise you create a new one and return that?

    sub do_something { my ( $normal_arg1, $normal_arg2, $optional_hashref ) = @_; my $hashref = $optional_hashref || { }; # # ...time passes... # $hashref->{ return1 } = "foo"; $hashref->{ return2 } = "bar"; return $hashref; } my %hash; my %newhash = do_something( $this, $that ); do_something( $this, $that, \%hash );

    -----------------------------------------------------
    Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
    "I can see my house from here!"
    It's not what you know, but knowing how to find it if you don't know that's important

Re: subroutines and namespace
by ariels (Curate) on Nov 02, 2001 at 19:27 UTC
    You can't (and, while not strictly crazy to want this, you are... misguided... or unwise...).

    The point about statically scoped (my) variables is that what they refer to can be completely determined by their point of definition. So if I say

    my $foo=17; sub quux { my $bar=29; $foo = baz($foo,++$bar) }
    then it is possible, just by examining the point of definition of quux, to determine what gets modified where.

    What you're after is global variables. You want some variable defined in one place in your program to get modified in an entirely different place. Why is this bad?

    Well, consider this code:

    use vars qw/$foo/; sub quux { my $bar=29; $foo = baz($foo,++$bar) } # ...code passes... $foo = 11; print quux($foo), "\n"; # ...more code... { local $foo = 39; print quux($foo), "\n"; }
    Note how every time some other $foo gets modified. Just by examining the new definition of quux you get no clues as to what gets modified. Instead, you have to examine the point where quux gets called, then work your way up the call stacks to surrounding blocks. You examine each block to see if local $foo sets up a new copy, or find $foo at the top level. And that's the copy of $foo that gets modified.

    Variable lookup with global variables is akin to spaghetti. You can eat it, but there are healthier alternatives...

      Yeah, but nothing else that takes 5 minutes to make tastes as good. (Maybe that's an analogy?)

      ------
      We are the carpenters and bricklayers of the Information Age.

      Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re: subroutines and namespace
by chromatic (Archbishop) on Nov 02, 2001 at 22:57 UTC
    So, my thought. I think it might be nice if there was a way to make a subroutine behave, in regard to naming scopes, exactly the same as if, instead of calling the module, you had cut and pasted the lines of code into your script in place of the subroutine call.

    I think there is, though it would involve some XS programming. If you can find the CV (internal data structure holding a subroutine) you want to duplicate, you could theoretically copy it and change its stash value (lexical scope, basically) and stick it in the appropriate GV (symbol table). Any XS hackers out there care to enlighten us?

    Barring that, is there a reason you can't use a closure?

    sub make_booking_SQL { my ($BookingID, $ref_booking, $ref_event, $ref_course) = @_; return sub { $sth = $dbh->prepare("SELECT * FROM booking WHERE BookingID = +$BookingID") or die $dbh->errstr; $sth->execute(); $ref_booking = $sth->fetchrow_hashref; $sth = $dbh->prepare("SELECT * FROM event WHERE EventID = $ref_ +booking->{'EventID'}") or die $dbh->errstr; $sth->execute(); $ref_event = $sth->fetchrow_hashref; $sth = $dbh->prepare("SELECT * FROM course WHERE CourseID = $re +f_booking->{'CourseID'}") or die $dbh->errstr; $sth->execute(); $ref_course = $sth->fetchrow_hashref; }; }
    You could also use placeholders and output binding, as in my DBI article. ;)
(tye)Re: subroutines and namespace
by tye (Sage) on Nov 02, 2001 at 22:55 UTC

    This is bad because getting the information to the subroutine requires that you correctly name the variables in the enclosing scope and there is no way to catch mistakes.

    It is also bad to have the subroutine tell the caller what variable names to use. What if I want to write some code that uses your subroutine twice? I can't have two different variables both called $ref_booking!

            - tye (but my friends call me "Tye")
Re: subroutines and namespace
by MZSanford (Curate) on Nov 02, 2001 at 16:34 UTC
    My only thought i getting code to work as is cut-and-pasted into place, would be to use Filter or, better yet Filter::Simple to actually manipulate the source code. This would let you litterally paste things into place. Now, let me say i do not suggest this. At one point wrote a macro style pre-processor thinking that, like in C, using code put into place as opposed to function calls would give the smallest time advantage. It did not work, the code was slower with the macro style functions, not to mention that editing your own source using a program is terribly un-clean, more so that goto even. but i would be interested to see if anyone else has had better luck.
    i had a memory leak once, and it ruined my favorite shirt.
Re: subroutines and namespace
by George_Sherston (Vicar) on Nov 02, 2001 at 21:05 UTC
    I'm most grateful to those who've troubled to kick in their $/50 or £/50 or whatever - perhaps even e/50 these days... whatever... it's been, if nothing else, an educational canter around some of the topics in this area.

    I take the points against my madcap scheme. But I'm not completely convinced (yet). Often one does want to keep the subroutine completely independent of the calling scope. But sometimes one (by which I mean "I" - and it may be this is a sign I'm getting other things wrong) have several different scopes that repeat the same, or roughly the same, bit of code. Just a little snippet, nothing fancy, but repeated a dozen times it begins to get heavy.

    And I know the canonical solution to this added weight is
    $RAM->buy('more')
    But at the same time, it wd just be nice to be able to write that snippet out once only. On the other hand, if I have to surround it with variable declarations and mechanisms for recovering variables, and if I have to pass it a whole load of arguments (and think about which arguments I'm passing) then it begins to be more trouble than it's worth.

    Likewise if the result is a proliferation of global variables, that's a bad idea - I'd like the variables in the subroutine to go out of scope when I leave the calling scope. And full-qualified variables are at least aesthetically unattractive.

    What I'd like is a pragma that I can use at the start of a package that merges that package's namespace with the namespace from which it was called. I don't know from perlguts, so this may be an impossible dream. And I'm still ready to be convinced it's a mad dream, or that it's a dream that is already a reality if I but knew how to do it. But it IS still my dream...

    § George Sherston
      On the other hand, if I have to surround it with variable declarations and mechanisms for recovering variables, and if I have to pass it a whole load of arguments (and think about which arguments I'm passing) then it begins to be more trouble than it's worth.

      This sounds like you're not factoring your functions enough, nor are you documenting your functions. If you design good functions (especially naming!!), then they will tell you how to call them. They will tell you what they need and how much of it they want. Don't throw the baby out with the bathwater, my brother. Try it for a month and see how you like it.

      ------
      We are the carpenters and bricklayers of the Information Age.

      Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

ERRATUM
by George_Sherston (Vicar) on Nov 02, 2001 at 16:31 UTC
    The eagle-eyed will have noticed that the last line in Get_Booking_Course_Event is missing. It should of course be:
    $dbh->disconnect;


    § George Sherston
Re: subroutines and namespace
by jepri (Parson) on Nov 03, 2001 at 08:59 UTC
    I suspect I have overlooked something major in the discussion here because when I was faced with (almost) the identical situation I just did:

    package whatever; ... @EXPORT = qw( $routine ); $routine = q^ $thing = something+something_else; lots and lots of code; ^; sub routine [ my ($thing, $other_thing) = @_; eval $routine; } package main; my $thing; my $other_thing; eval $routine;

    If you want to run routine in local scope, eval $routine it. If you don't want it messing with your variables, call it with routine()

    ____________________
    Jeremy
    I didn't believe in evil until I dated it.

      Warning. That eval solution needs error checks. Else you could get very confused working out why you are getting no result, when the problem is that $routine expects to be sharing some variables with its environment, the environment doesn't have them, and strict finds this to be a situation worth complaining about.

      (Just to list one non-obvious way that a non-trivial error in $routine may need to be handled.)

      UPDATE
      Re the reply, I understand fully. I just like pointing these things out because you have to expect people to just lift code from here directly without understanding the issues. So if you are posting it as a suggestion, it should look like production code.

      That said, I find the eval solution painful on the face of it. I would prefer to solve that with just using Filter::Simple and a few well-chosen macros. If I was going to solve things that way. Which I would probably avoid for reasons listed above...

        The perils of doing an off-the-cuff answer to a post. When I implemented it I eventually had buckets of error checks in the routine itself (e.g.  if defined) and checked the results with an or die "\$routine failed with error $@". I also had almost every second line printing into a log file (my personal favourite way of documenting code).

        But after that there's not much difference between the evals and real subs except some faulty code can stomp all over variables in your current namespace.

        ____________________
        Jeremy
        I didn't believe in evil until I dated it.

Re: subroutines and namespace
by hsmyers (Canon) on Nov 05, 2001 at 03:59 UTC
    hmmm… I'm not real clear on why you would want a sub to use stuff without stuff being global but
    #!/perl/bin/perl # # -- use strict; use warnings; use diagnostics; my $test = "hello world\n"; my $stub = sub {$test = "goodbye world\n"}; print $test; &$stub; print $test;
    at least seems to work… This is common in Lisp and other 'functional' languages, and Pascal if memory serves, but I can't say as to how I've ever felt the need in other languages. Dosen't mean that there isn't a need, just not one I've ever run into.

    hsm