dragonchild has asked for the wisdom of the Perl Monks concerning the following question:

After shaving a yak, I ended up in Hash::Util, specifically in lock_keys(). Noticed heavy use in the PurePerl version of Internals, specifically Internals::SvREADONLY(). Never having heard of it, I looked it up. Lo and behold, although SvREADONLY() was used by the XS, it wasn't exposed by the XS. Bopping around a bit, I found use of a bunch of functions in the Internals:: namespace that aren't exposed by Internals. And, furthermore, it seems like these functions are available without use-ing Internals itself.

So, the question - what's in Internals:: and where can I get some documentation on it? Sounds like all sorts of goodies are in there and I wanna get me some of that! :-)

Update: Per Anonymonk's reply, apparently a whole bunch of seemingly-unrelated things are provided in universal.c and that's just the list for 5.8.9. In 5.10.0's universal.c, there's twice the list. (Scroll down halfway looking for /^XS\(/). Why are these functions all stuffed together in universal.c? Where are they documented? If they're exposed, I would think they would be documented somewhere.

My criteria for good software:
  1. Does it work?
  2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Replies are listed 'Best First'.
Re: Will the real Internals module please stand up?
by ikegami (Patriarch) on Mar 01, 2009 at 05:28 UTC 
    >perl -wle"while(($n,$g)=each %Internals::) { print $n if *$g{CODE} }"
hash_seed
hv_clear_placeholders
HvREHASH
rehash_seed
SvREADONLY
SvREFCNT

    [download]

    ActivePerl v5.10.0 build 1001 built for MSWin32-x86-multi-thread

    Update: 

    #!perl -wl
my @todo = *::;
while (@todo) {
   my $st = shift(@todo);
   my @sl = sort values %$st;
   print for map /^\*(.*)/, grep *$_{CODE}, @sl;
   push @todo, grep !/\*main::main::/, grep /::$/, @sl;
}

    [download]

    ActivePerl v5.10.0 build 1001 built for MSWin32-x86-multi-thread:

    Read more... (2 kB)

    ActivePerl v5.8.8 build 824 built for MSWin32-x86-multi-thread

    Read more... (1143 Bytes)

    This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi: 

    DynaLoader::boot_DynaLoader
Internals::HvREHASH
Internals::SvREADONLY
Internals::SvREFCNT
Internals::hash_seed
Internals::hv_clear_placeholders
Internals::rehash_seed
PerlIO::get_layers
Regexp::DESTROY
UNIVERSAL::VERSION
UNIVERSAL::can
UNIVERSAL::isa
attributes::bootstrap
utf8::decode
utf8::downgrade
utf8::encode
utf8::is_utf8
utf8::native_to_unicode
utf8::unicode_to_native
utf8::upgrade
utf8::valid
PerlIO::Layer::NoWarnings
PerlIO::Layer::find

    [download]
[reply]
[d/l]
[select]
      Only code there 
      $ perl -le'print for %Internals::'
SvREFCNT
*Internals::SvREFCNT
hv_clear_placeholders
*Internals::hv_clear_placeholders
hash_seed
*Internals::hash_seed
SvREADONLY
*Internals::SvREADONLY
HvREHASH
*Internals::HvREHASH
rehash_seed
*Internals::rehash_seed

      [download]
[reply]
[d/l]
        How is that better? The output is less readable and it introduces an assumption.
[reply]
[reply]
Re: Will the real Internals module please stand up?
by Anonymous Monk on Mar 01, 2009 at 05:31 UTC
[reply]

      Why isn't it documented? How official is this? For example, can I rely on 

      our $VERSION = version::qv('1.0.0');

      [download]

      without doing use version; first? (Ignore compatibility with 5.8.)

[reply]
[d/l]
[select]
        I don't know, appears to be a case of "some patch got accepted"
[reply]
        can I rely on our $VERSION = version::qv('1.0.0'); without doing use version; first?
        In counter-example, the utf8 and Win32 doc specifically say what can be assumed to be available without loading the module, so unless you can find a similar guarantee in the version doc, I think you'd be overly trusting to do so.
        --
        Online Fortune Cookie Search
        Office Space merchandise
[reply]
Re: Will the real Internals module please stand up?
by ysth (Canon) on Mar 01, 2009 at 19:39 UTC
    Undocumented things in universal.c should not be depended on; they should only be used by core modules. They aren't documented on purpose, to allow them to be changed whenever and however necessary. For those purposes, the code is good enough documentation.
    --
    Online Fortune Cookie Search
    Office Space merchandise
[reply]

      Some are documented

      UNIVERSAL::isaimplied
      UNIVERSAL::canimplied
      UNIVERSAL::DOESimplied
      UNIVERSAL::VERSIONimplied
       
      version::newundocumented availability
      version::qvundocumented availability
      version::booleancompletely undocumented
      version::vcmpcompletely undocumented
      version::noopcompletely undocumented
      version's UNIVERSAL::VERSION overrideundocumented availability
       
      utf8::is_utf8documented
      utf8::validdocumented
      utf8::encodedocumented
      utf8::decodedocumented
      utf8::upgradedocumented
      utf8::downgradedocumented
      utf8::native_to_unicodecompletely undocumented
      utf8::unicode_to_nativecompletely undocumented
       
      Internals::SvREADONLYcompletely undocumented
      Internals::SvREFCNTcompletely undocumented
      Internals::hv_clear_placeholderscompletely undocumented
      Internals::hash_seedcompletely undocumented
      Internals::rehash_seedcompletely undocumented
      Internals::HvREHASHcompletely undocumented
       
      PerlIO::get_layersundocumented availability
       
      re::is_regexpundocumented availability
      re::regnameundocumented availability
      re::regnamesundocumented availability
      re::regnames_countundocumented availability
       
      Tie::Hash::NamedCapture::undocumented availability

      The table applies to 5.10.0. I omitted methods except those in UNIVERSAL (since those have a global effect).

      Legend:

      documentedThese functions are documented as always available.
      impliedThe documentation for these functions implies they are always available.
      undocumented availabilityThese functions are documented, but their availability isn't.
      completely undocumentedThe documentation of the modules for these functions don't mention them.

      Interestingly, version claims to override UNIVERSAL::VERSION, but there is only one function. The "override" is always in effect.

      Interestingly, the following bindings are absent:

      • re::regmust
      • re::regexp_pattern
[reply]
[d/l]
[select]

        What do you mean about "bindings are absent"?

        Most of this stuff, in particular availability, is not documented directly for a reason. So for instance Tie::Hash::NamedCapture is used to implement named captures. It is loaded automagically by the internals when you do a $+{foo} style lookup. It uses a bunch of the re:: functions, which are exposed via universal because they are then compiled into the core executable and thus do not require an additional library to be loaded. Now I could have implemented the entire thing without using any perl level modules or exposing any subs, but that would have limited the options available to the programmer which I felt was not the Perl way.

        ---
        $world=~s/war/peace/g

[reply]
Re: Will the real Internals module please stand up?
by ikegami (Patriarch) on Mar 01, 2009 at 19:49 UTC
    The utf8:: functions can't be loaded by use utf8; since that's a pragma. I guess universal.c was a speedier alternative to making a module just to expose the utf8:: functions.
[reply]
[d/l]
[select]
      They could have been included in the module, the same way that there are utility functions in the charnames pragma that you can load without triggering the pragma behavior via "use charnames ()" or "require charnames".

      So either it was just a "speedier alternative", as you say, or it was desired that the module not need to be loaded. I can't guess which.

      --
      Online Fortune Cookie Search
      Office Space merchandise
[reply]

        One reason to put things in universal.c is it allows you to use these functions in miniperl. If they are used by things that are run by miniperl or they are used to test perl then they need to be available early. Another reason is that functions are used to implement parts of perls internals, and therefore need to be around. An example is some of the re:: routines that are used to implement named captures. They need to be usable without loading the re dll, which also contains a full debugging enabled version of the regex engine.

        ---
        $world=~s/war/peace/g

[reply]

Back to Seekers of Perl Wisdom