Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Can a Perl module "inspect" itself and provide details about methods and parameters?

by Br'er Rabbit (Novice)
on Sep 21, 2019 at 20:34 UTC ( #11106512=perlquestion: print w/replies, xml ) Need Help??

Br'er Rabbit has asked for the wisdom of the Perl Monks concerning the following question:

I'm suffering through an exercise to do some image processing with ImageMagick by way of PerlMagick (the Image::Magick Perl module).

The documentation is maddening (notwithstanding RLS's praise of it as "unimpressive"), and anyone who wants to do anything other than a simple example, will find theirself roundtripping from pillar to post.

Until reading his comment, I hadn't thought to look at the--shudder--source code,

...but my thoughts were running a different direction.

I seem to remember that Perl modules are "objects" or collections of them and I remember from the many attacks on my comprehension by "object-orientation" that "objects" can be ASKED this-or-that. (It's supposed to be a "feature.")

I also recall that they implement METHODS which can be public or private to make answer to queries.

And so...

...SINCE "objects" and "Perl modules" are way smarter than me, knowing way much more about their innard-parts than me, I would like to ask them:

"Hey, could you please TELL me what PUBLIC methods you have, and WHAT parameters they use, including their DEFAULTS?"

Kinda like: "Hey, why don't you read your own damn source code and highlight the most important bits for me?"

If they would just do that, I would write their answers down and tape them to the stupid monitor and stop all this hunting around.

Thank you for your time. Peace.

  • Comment on Can a Perl module "inspect" itself and provide details about methods and parameters?

Replies are listed 'Best First'.
Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by stevieb (Canon) on Sep 21, 2019 at 21:50 UTC

    I think it'd be a futile effort to try to sort out what params each subroutine accepts because they can come in and be handled in a myriad of ways.

    However, using my Devel::Examine::Subs, you can at least get info about subroutines within a file, module or directory.

    Here's a code snip. In this case, I'm in a distribution directory (my Mock::Sub repository):

    use warnings; use strict; use Devel::Examine::Subs; my $des = Devel::Examine::Subs->new(file => '.'); my $data = $des->all; for my $file (keys %$data){ print "$file:\n"; for my $sub (@{ $data->{$file} }){ print "\t$sub\n"; } }

    Output:

    lib/Mock/Sub/Child.pm: new _mock remock unmock called called_count called_with name reset return_value side_effect _check_side_effect mocked_state _no_warn DESTROY _end lib/Mock/Sub.pm: import new mock mocked_subs mocked_objects mocked_state DESTROY __end

    You can even get info about each sub. This example is reading just a single file (but it does work with directories and modules, it's just a bit different to extract the data. Docs contain this info.

    use warnings; use strict; use feature 'say'; use Devel::Examine::Subs; my $des = Devel::Examine::Subs->new(file => 'lib/Mock/Sub.pm'); my $subs = $des->objects; for my $sub (@$subs){ say $sub->name; say "\tstart line: ". $sub->start; say "\tend line: " . $sub->end; say "\tline count: " . $sub->line_count; # the following would list out the entire code block # for the sub. Commented out for output brevity #say "\tsub code:"; #say "\t\t$_" for @{ $sub->code }; }

    Snip of output:

    import start line: 14 end line: 17 line count: 4 mocked_objects start line: 74 end line: 82 line count: 9 new start line: 18 end line: 26 line count: 9 DESTROY start line: 100 end line: 101 line count: 2 __end start line: 102 end line: 102 line count: 1 mocked_state start line: 83 end line: 99 line count: 17 mocked_subs start line: 62 end line: 73 line count: 12 mock start line: 27 end line: 61

    Works on in-memory modules as well. Send the module name to the file parameter. If the module is installed, we'll load it and read from there:

    use warnings; use strict; use Devel::Examine::Subs; my $des = Devel::Examine::Subs->new(file => 'Logging::Simple'); my $subs = $des->all; print "$_\n" for @$subs;

    Output from my Logging::Simple distribution module:

    BEGIN _sub_names new level file name timestamp levels labels display print child custom_display fatal _generate_entry _levels _log_only

    Note that the _end() methods are simply placeholders; they allow my editor to fold all of my POD up into the last subroutine so when I go to bottom of the current file, it doesn't take me to the end of my POD.

    You could also use PPI to do the same thing. That's what my distribution uses, and added a whole slew of things around it.

      @Stevieb

      > I think it'd be a futile effort to try to sort out what params each subroutine accepts because they can come in and be handled in a myriad of ways.

      This part is just sad: if objects were self-documenting, then they wouldn't have to rely on the existence of (good) external documentation.

      "If I ever write one", (he wrote, pompously) "I'll sure gladly provide everything everybody would want to know."

      Your code will nevertheless be a Big Help in filling in some holes about What's Actually Callable.

      Thank you.

        "This part is just sad: if objects were self-documenting, then they wouldn't have to rely on the existence of (good) external documentation."

        When I'm writing against an object, my IDE pops up with a list of valid methods to choose from, and autocompletes if I desire. To boot, once I've typed in the method name, it'll display to me what arguments the method (or function) accepts. Now, if the function/method is accepting a single hash for example, there's no way to know at all what key/value pairs are used within that hash. However, if all of the parameters are individual scalars in a specific order, it will list those (with their names included).

        Here's an image showing the IDE displaying a list of available methods to an external class (works on objects all the same).

        ...and here's an image showing the parameters for an object's method.

        A good IDE can help tremendously, especially when you're working with numerous classes at the same time. You can CTRL-click on any variable, method, function or anything, and you can make one click to go to the definition of it immediately.

        Rarely do I need to view documentation online, or the command line. Typically in Perl, we gather up all of our parameters at the top of each subroutine, so being able to go directly to the definition of a sub makes it trivial to see exactly how params are being used.

        With ImageMagic—since that was the genesis of the thread(s)—you have an added complication. You have to document, introspect the Perl interface/bindings and the C libs beneath. Documentation and code are not, and can never be, the same any more than even simple math could be. I don’t think it’s particularly sad. It is what it is. Perl’s deep flexibility, including lack of types, also precludes a perfect solution, maybe even a consistently decent one, to this kind of introspection. Temporarily suspending the excellent, at least interesting, ideas of affordance and literacy in code; a tool that is also its own manual will end up cumbersome and awkward as both. XML and XSLT come to mind for me here.

        @AllMonksWhoTookTheTime

        Thank you, esp. for explaining what must seem obvious to you by now. (I hope this thread survives: at my age, I'm sure I'll be back to read it... ;^) )

        Note @Stevieb: Re: Devel::Examine::Subs had to install a good number of other modules first, so out-of-touch was my CPAN, and now am only failing t/42 and t/43. Clearly, from you own observation, presaging others', about the "uncertainty" associated with retrieved parameters, the pressure to have and use the tool is off. I'll give it a few more good tries, but will otherwise take the hint and move on and look for my answers in the source. Thank you.

Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by hippo (Bishop) on Sep 21, 2019 at 22:27 UTC
    I seem to remember that Perl modules are "objects" or collections of them

    This isn't necessarily the case. Some Perl modules are object-orientated and are classes (not "objects" which are things instantiated from the classes) but equally some Perl modules are not object-orientated at all. Some (like the venerable CGI.pm) can be treated either as a class or not, depending on the calling code by having both an OO interface and a procedural interface within the same module.

      WiringPi::API is exactly like that. It has both an Object Oriented interface, as well as a procedural interface, and both interfaces can be used at the same time, if desired.

Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by GrandFather (Saint) on Sep 21, 2019 at 22:44 UTC

    Out of the box Perl provides very light weight support for OO. As with most things Perl you can use it to create pretty much whatever you need. So in this specific case it comes down to "how was Image::Magick written" and for the most part the answer is "not in Perl". The Perl code is a fairly thin wrapper around the underlying library so even in the best case introspection of the Perl code isn't going to be a great help.

    But it's worse than that. Even languages that provide introspection can't tell you much about semantics. Sure, you may be able to find out about methods that can be called and maybe even about the parameters they are expecting, but there is a large gap between that an knowing what those parameters mean, or how they interact, or the order methods should be called or a huge range of other vital stuff you need to know to use an API - OO or otherwise.

    You may find it more helpful to ask for help than bemoaning a lack of good documentation and trying to run an endgame around the documentation issue.

    Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

      Good point on not all Perl code is actually written in Perl. That's definitely something that I should have clued in on considering half the "Perl" code I've written the last two years has been C, C++ and XS :)

Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by swl (Vicar) on Sep 22, 2019 at 22:02 UTC

    Just a couple of data points, as others have discussed the overall issues with XS, AUTOLOAD and the like. Neither will process argument lists, so they only address part of the question.

    Devel::Symdump will give a huge amount of information about packages, but I prefer Class::Inspector these days as it is simpler to use.

    use Devel::Symdump; my $obj = Devel::Symdump->rnew( $package_name ); my @methods = $obj->functions(); # if you are sure private methods only start with underscore... my @public_methods = grep {$_ !~ /^_/} @methods;

    and

    use Class::Inspector; my $methods = Class::Inspector->methods( $package_name ); my $public_methods = Class::Inspector->methods( $package_name, 'public +' );

      Fun. Play stub–

      use 5.10.0; package DoReMe { use Moo; use namespace::autoclean; # Redacts introspection output significa +ntly! has "solfege" => is => "lazy"; } use Class::Inspector; my $methods = Class::Inspector->methods("DoReMe"); say join $/, @{$methods}; say "--"; use Devel::Symdump; my $obj = Devel::Symdump->rnew("DoReMe"); say join $/, $obj->functions;
      BUILDALL BUILDARGS DEMOLISHALL does meta new solfege -- DoReMe::new DoReMe::solfege
      Cool! Using a regex stolen from Class::Inspector:

      CORE functions:

      perl '-MData::Dumper()' -e '@_=sort grep {/\A[^\W\d]\w*\z/o} keys %{"$ +{CORE}::"};print Data::Dumper::Dumper\@_'
      CORE + Cwd import:
      perl '-MData::Dumper()' -MCwd -e '@_=sort grep {/\A[^\W\d]\w*\z/o} key +s %{"${CORE}::"};print Data::Dumper::Dumper\@_'
      Everything:
      perl '-MData::Dumper()' -e '@_=sort keys %{"${CORE}::"};print Data::Du +mper::Dumper\@_'
        > CORE functions:

        nope these are some plain named symbols in the %main:: STASH since your var $CORE is undef.

        Anyway you need to check the {CODE} slot of the glob-values to be sure that they hold a code-ref.

        DB<30> sub test { "bla" } DB<31> x map { \&$_ } grep {defined &$_ } values %:: 0 CODE(0x3477178) -> &main::test in (eval 40)[C:/Perl_524/lib/perl5db.pl:737]:2-2 1 CODE(0x3463838) -> &main::dumpvar in C:/Perl_524/lib/dumpvar.pl:474-501 2 CODE(0x3457a78) -> &main::dumpValue in C:/Perl_524/lib/dumpvar.pl:33-40 DB<32>

        dumpvar and dumpValue are internal debugger routines

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by haukex (Bishop) on Sep 21, 2019 at 23:57 UTC
    I'm suffering through an exercise to do some image processing with ImageMagick by way of PerlMagick (the Image::Magick Perl module). The documentation is maddening (notwithstanding RLS's praise of it as "unimpressive"), and anyone who wants to do anything other than a simple example, will find theirself roundtripping from pillar to post.

    Just as a note, you might be interested in How Geizhals took out ImageMagick before ImageMagick took *us* out.

      Sidebar on the same topic: GraphicsMagick is, or at least was, a safer and more performant fork of ImageMagick. It’s been several years since I knew that to be a fact however so take it with a pinch of kosher.

Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by LanX (Sage) on Sep 22, 2019 at 17:13 UTC
    Yes and no.

    No ...Perl is so dynamic that you will hardly automatically parse all subs without an AI which is more clever than you.

    But Yes ... you might built a solution which fits in 80-90 percent of the cases.

    For instance is it allowed to dynamically proxy subs with AUTOLOAD and the way args are read from @_ can vary in many ways.

    In the case of OO classes you'd also need to check the inheritance tree in @ISA.

    And I don't even know which backdoors XS modules are allowed to use.

    Saying so,after using a modul it's possible to dynamically introspect its Symbol Table Hash aka STASH.

    And it's possible to list all lines involving the @_ array.

    Something standard like my ($self,$name) = @_ might be easy to decipher.

    But this will only show you the internal variable name and not the description of an argument $name.

    Hence this would only be a semi automatic help often requiring human intervention.

    But in your case, no way

    From my experience with Image Magick this won't help much though, because the underlying C library is pretty inconsistent and Perl might only provide a thin layer.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

Re: Can a Perl module "inspect" itself and provide details about methods and parameters? -- oneliner
by Discipulus (Abbot) on Sep 22, 2019 at 16:13 UTC
    Hello Br'er Rabbit,

    > Can a Perl module "inspect" itself and provide details about methods..

    Yes! You can parse a perl document with PPI two oneliners just to get started:

    # a bare package provided as string perl -MPPI -e "print $_->content for @{ PPI::Document->new(\'package M +yPack{ sub new{ bless{}, shift} }')->find('PPI::Statement::Sub') }" #output sub new{ bless{}, shift} # inspecting Data::Dump module (getting its path from %INC) perl -MPPI -MData::Dump -e "print $_->content for @{ PPI::Document->ne +w( $INC{'Data/Dump.pm'} )->find('PPI::Statement::Sub') }; " #output sub dump { local %seen; local %refcnt; local %require; local @fixup; ...

    You can go further trying to extract arguments received (good luck ;) and maybe you have to learn what PPI PDOM Tree is:

    perl -MPPI -MPPI::Dumper -e "PPI::Dumper->new( PPI::Document->new(\'pa +ckage MyPack{ sub new{my class shift; return bless{},$class} }') )-> +print" PPI::Document PPI::Statement::Package PPI::Token::Word 'package' PPI::Token::Whitespace ' ' PPI::Token::Word 'MyPack' PPI::Structure::Block { ... } PPI::Token::Whitespace ' ' PPI::Statement::Sub PPI::Token::Word 'sub' PPI::Token::Whitespace ' ' PPI::Token::Word 'new' PPI::Structure::Block { ... } PPI::Statement::Variable PPI::Token::Word 'my' PPI::Token::Whitespace ' ' PPI::Token::Word 'class' PPI::Token::Whitespace ' ' PPI::Token::Word 'shift' PPI::Token::Structure ';' PPI::Token::Whitespace ' ' PPI::Statement::Break PPI::Token::Word 'return' PPI::Token::Whitespace ' ' PPI::Token::Word 'bless' PPI::Structure::Constructor { ... } PPI::Token::Operator ',' PPI::Token::Symbol '$class' PPI::Token::Whitespace ' '

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
Re: Can a Perl module "inspect" itself and provide details about methods and parameters?
by Anonymous Monk on Sep 26, 2019 at 20:36 UTC
    I seem to remember that Perl modules are "objects" or collections of them and I remember from the many attacks on my comprehension by "object-orientation" that "objects" can be ASKED this-or-that. (It's supposed to be a "feature.")

    Sometimes you can learn about objects by dumping them:

    perl -MHTTP::Tiny -MData::Dumper -le '$o=HTTP::Tiny->new;print Dumper +$o' $VAR1 = bless( { 'timeout' => 60, 'keep_alive' => 1, 'max_redirect' => 5, 'verify_SSL' => 0, 'agent' => 'HTTP-Tiny/0.070', 'no_proxy' => [] }, 'HTTP::Tiny' );
    Ish:
    perl -MHTTP::Tiny -le '$o=HTTP::Tiny->new;print "$_ = $o->{$_}" for so +rt keys %$o' agent = HTTP-Tiny/0.070 keep_alive = 1 max_redirect = 5 no_proxy = ARRAY(0x7fc821802fc8) timeout = 60 verify_SSL = 0
    Sometimes, there's no params:
    perl -MImage::Magick -MData::Dumper -le '$o=Image::Magick->new;print D +umper $o' $VAR1 = bless( [], 'Image::Magick' );

      This "works", but be very aware you're breaking a fundamental tenet of OO (encapsulation) by doing so.

      And by so sinning you're lining yourself up for the inevitable great vengance and furious anger which will rain down upon you (c.f. Ezekiel 25:17) when the module author decides that mauve has more RAM and changes to inside-out objects in a subsequent release and whatever hack you came up with depending on the prior internal implementation breaks.

      Fine for pedagogical exploration; never do this in real code.

      The cake is a lie.
      The cake is a lie.
      The cake is a lie.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11106512]
Approved by GrandFather
Front-paged by johngg
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (5)
As of 2022-01-18 21:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:












    Results (54 votes). Check out past polls.

    Notices?