Beyond Inside-Out

The inside-out techique of class construction has shown that inheritance in object oriented Perl can work as smoothly as it does in dedicated OO languages. Unlike traditional Perl classes, one or more inside-out classes can be made base classes of any other Perl class, inside-out or not. The other way around, any class can become a base class of an inside-out class, though only one of the base classes can be non-inside-out.

The price for this freedom is also well known. Inside-out classes are not automatically garbage-collected and thread-cloned. Each class must take care of that itself -- there is no general way of doing that because only the class knows which data it defines. A related problem, less often discussed but also serious, is serialization. Whether for viewing or for object persistence, the usual tools (Data::Dumper, Storable) know nothing about the object's data in inside-out storage. A class can easily provide the necessary hooks (STORABLE_freeze and STORABLE_thaw, for instance), but each class must do so individually.

There are two essentials in the working of inside-out classes that are key to the freedom of inheritance:

- Each class can freely define the data it wants to associate with an object, independent of what other classes think the object contains.

- No method of an inside-out class accesses (de-referrences) the body of its objects directly.

These essentials can be realized in a different way. If every object had, for every potential class, an individual reference to present to that class instead of its body, both conditions would be met. Each class would have an individual set of data associated with each object. To access the data, it would de-reference the object's alter ego and not the body itself.

To show that the concept works I have put together a module named Alter, which exports a function named ego(). ego() can be used on every reference (object) to store and retrieve an alternative reference which depends on the caller of ego(). Thus every class can establish its individual view of and object, independent of the views of other classes.

Here is a trivial class that implements a single accessor method. It doesn't demonstrate the salient features of Alter::ego but shows how the implementation of a standard class would look like. Essentially it works like a standard (not inside-out) Perl class, except that all methods invoke the object's Alter::ego before accessing its data.

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;

my $x = MyClass->new( 'Otto');
print $x->name, "\n";
$x->name = 'Erich';
print $x->name, "\n";
exit;

########################################################

package MyClass;
use blib;
use Alter qw( ego);

sub new {
    my $class = shift;
    bless( \ my $o, $class)->init( @_);
}

sub init {
    my $ob = shift;
    ego $ob, { name => shift }; # set up alter ego as a hash ref
    $ob;
}

sub name :lvalue { ego( shift)->{ name} } # access the alter ego
__END__
[download]

A preliminary implementation of Alter contains a (slightly) more extensive example. It is available here. The pod can be viewed there too. It's an XS module, so you need the right compiler to make it run. It is not extensively tested, but should be workable. Needless to say -- don't base any serious code on it, it's nothing but a proof of concept.

Anno

Comment on Beyond Inside-Out Download Code

Replies are listed 'Best First'.
Re: Beyond Inside-Out by kyle (Abbot) on May 29, 2007 at 14:06 UTC
It seems as thought Alter::ego could be implemented in an inside-out class itself. `package Alter; use Class::Std; my %ref_of :ATTR; sub ego { my $self = shift; if ( @_ ) { $ref_of{ident $self} = shift; } return $ref_of{ident $self}; }` [download] I looked for a test suite in your .tar.gz to show me the error of my ways, but it seems all it does is `use_ok`. Implementing in terms of Class::Std reopens the thread cloning problem, but I think Object::InsideOut takes care of that. Since I haven't had to deal with threading problems, I haven't used it myself. Is a pure Perl implementation not possible? Have I missed some important feature of the module?	[reply] [d/l] [select]
Re^2: Beyond Inside-Out by Anno (Deacon) on May 29, 2007 at 14:55 UTC
Yes, I believe an implementation of `Alter` in terms of an inside-out class would be possible. From my perspective that would be a rather roundabout way. You'd require one call to `ident()`plus a hash acces to retrieve the alter ego. More importantly, my goal is not to have to support garbage collection and thread cloning in the first place, but let Perl do it. These problems can be solved, and have been. I've done it myself in my module Hash::Util::FieldHash. I think it's far better not to have them in the first place. I would like a pure Perl implementation, and I'm trying to keep the necessary XS part to a minimum, but I don't see one. Magic has the unique combination of being out of band to Perl code, but being serviced by garbage collection and thread cloning. Magic needs an XS part. Anno Update: I forgot to mention: No there isn't a set of tests yet with Alter.pm. It is entirely preliminary.	[reply] [d/l] [select]
Re: Beyond Inside-Out by shmem (Chancellor) on May 29, 2007 at 15:12 UTC
Wow. This is so simple, elegant and such an obvious way to set up inside out objects (once you have seen it, that is :-), that I wonder why it wasn't done before. I'd like to see that as the standard implementation of inside-out objects in the perl core, but at least up on CPAN. --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply]
Re^2: Beyond Inside-Out by Anno (Deacon) on May 29, 2007 at 16:10 UTC
Thanks for your kind words. ... such an obvious way ... It is quite obvious once it is understood what needs to be done to make Perl objects play nice with inheritance. From my perspective, the merit goes to the original inside-out implementation where these requirements are met in an ingenious pure-perl way. Doing it by magic is comparatively easy :) Anno	[reply]
Re^3: Beyond Inside-Out by shmem (Chancellor) on May 29, 2007 at 20:36 UTC
Your merit is simplifying, so that one hasn't to load lengthy Modules or fiddle with lexically scoped hashes and DESTROY subroutines and the like, thus making inside out classes really robust, transparent and elementary. It's all falling into place now, for me. Thanks for this great meditation. --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply]
Re: Beyond Inside-Out by clinton (Priest) on May 30, 2007 at 08:51 UTC
I would like to check that I have understood your technique properly. Do I have this correct? The purpose of this module is to: Allow arbitrary subclassing of any object without interfering with its super or subclasses Only allow access to properties via methods, instead of accessing them directly via (eg) hash keys Automate garbage collection You do this by: Accepting any type of object to new() Taking the 'identity' of that object and, within `MyClass` (and within any other Alter Ego style class), associating that identity with (in your example) an anonymous hash, which will be used to contain the actual data relevant to MyClass That anonymous hash is added to a bigger anonymous hash (the corona) which contains all the (Alter Ego) properties for this object: the key is the classname of the caller, the value is the created anon hash with the values relevant to that class: `$corona = { # Alter ego for properties from ClassA ClassA => { foo => 'foo', bar => 'bar' }, # Alter ego for properties from ClassB ClassB => { baz => 'baz', foo => 'foo' # Doesn't interfere with t +he foo property from ClassA } }` [download] To get/set a property that belongs to this class, you HAVE TO call a method which finds the alter ego (the associated anonymous hash) for the class in which the method lives (as determined by caller), and then that hash is used by the method to do the required work. The obvious benefits are: The properties belonging to a particular class cannot interfere with the properties from a super/sub-class It doesn't matter what type the original object is - any object can be subclassed It is fast and not convoluted It catches run-time typos when trying to access properties (except in the methods which live in the class itself) Questions Tye's concern is that caller does not always return what we expect, because, although we may be calling `package1::foo`, that method could be aliased to `package2::bar`. As far as I can see, that should not matter, because the value associated with `foo` (handled by `bar`) is still encapsulated, and so cannot be interfered with. There are other problems with this (see Using SUPER in dynamically generated subs), but I don't see the issue with this particular question. What am I missing? I don't understand why garbage collection works? Why, when `$object` goes out of scope will the `corona` go out of scope too? As far as I can see, you're not storing a reference to the `corona` in the `$object`. You mention of problem of serialization. Your solution doesn't help with this - you still need to provide Storable hooks for that, no? I hope I've understood this correctly - and corrections appreciated thanks, clint	[reply] [d/l] [select]
Re^2: Beyond Inside-Out by Anno (Deacon) on May 30, 2007 at 11:33 UTC
The purpose of this module (`Alter`) is to: Allow arbitrary subclassing of any object without interfering with its super or subclasses Yes. Only allow access to properties via methods, instead of accessing them directly via (eg) hash keys No. Unlike inside-out, `Alter` doesn't organize object data in hashes keyed by id. Each class sees the object as a single reference (type of its choice). It's up to the class to organize the data further. Automate garbage collection Yes. And thread-cloning. The mechanism is actually even simpler than your sketch. Assume there is a function of a single variable `_corona($obj)`. For each object it produces a single hash(ref) that is autovivified empty on the first call, and stays with the object for its lifetime. This function is not caller-sensitive. The caller-dependent behavior of `Alter::ego()` can then be described in Perl (code untested): `package Alter; sub ego { my $obj = shift; my $corona = _corona($obj); my $class = caller; return $corona->{ $class} if exists $corona{$class} ); $corona{ $class} = shift \|\| {}; }` [download] Thus the mechanism is: Equip each object (magically) with one hidden hash, its corona. Use that hash, keying by class name, to store the individual alter ego of the object for each class. The technique does nothing to catch misspelled attribute names, it doesn't know of attributes. It brings back the (good or bad, but old) days when "an object is nothing but a reference", only this time an individual one for every class. To your questions, skipping the one about Tye's concern. I don't understand why garbage collection works... Garbage collection and thread cloning are magic-aware, that's "all" there's to it (which is a lot). That makes magic a good choice for object data. It's out of band to normal Perl code (like the id-keyed hashes of inside-out), but Perl knows about magic and deals with it. You mention of problem of serialization. Your solution doesn't help with this - you still need to provide Storable hooks for that, no? Yes, you need hooks, but with `Alter` it is easy to provide one generic set of hooks that any class can inherit or even import. All it has to do is give the corona (a normal Perl hash) to the dumper, or to restore the corona of a given object. The equivalent is harder to do with inside-out objects where the object itself (all the dumper gets to see) bears no traces of its data. It's like this: Primitive old-style objects tatoo their data on their skin, one tatoo over the other if classes demand. The highly sophisticated inside-out objects store their data in bank vaults on small lexical islands, each in a different class. Hard to trace in case of a sudden death. Alter-stype objects collect their data in a secret pouch with neat compartements for each class, always together, but easy to keep apart. Anno	[reply] [d/l] [select]
Re^3: Beyond Inside-Out by kyle (Abbot) on May 30, 2007 at 16:29 UTC
Thus the mechanism is: Equip each object (magically) with one hidden hash, its corona. Use that hash, keying by class name, to store the individual alter ego of the object for each class. This reminds me of another "make inheritance work" technique I'd heard of. Every object is a hash ref (as usual), but every class has their own key into it. Basically every object looks like the `$corona` in clinton's description. The `Alter` advantage over that method is mainly privacy. The various classes sharing space can't meddle with each other's attributes. If `ego` is changed to address tye's concern (so that a class can specify its own name), I'm guessing that `Alter` loses that advantage too. At that point, you might as well write a base class with `ego` in it.	[reply] [d/l] [select]
Re^4: Beyond Inside-Out by Anno (Deacon) on May 31, 2007 at 14:32 UTC
Re^5: Beyond Inside-Out by kyle (Abbot) on May 31, 2007 at 15:43 UTC
Re^3: Beyond Inside-Out by clinton (Priest) on May 30, 2007 at 11:40 UTC
Thanks Anno, that makes a lot of sense I would be interested to understand more about Tye's concerns and how valid they are.	[reply]
Re^4: Beyond Inside-Out by Anno (Deacon) on May 30, 2007 at 12:56 UTC
Re^5: Beyond Inside-Out by xdg (Monsignor) on May 30, 2007 at 13:14 UTC
Some notes below your chosen depth have not been shown here
Re: Beyond Inside-Out (fluff) by tye (Sage) on May 29, 2007 at 13:27 UTC
I'm disappointed that you chose to leave out the heart of your idea, how ego() works. - tye	[reply]
Re^2: Beyond Inside-Out (fluff) by Anno (Deacon) on May 29, 2007 at 14:26 UTC
I'm disappointed that you chose to leave out the heart of your idea, how ego() works. Sorry. That can be helped :) `ego($obj, $alter)` sets the alter ego of `$obj` to be `$alter`. When called with one argument, `ego($obj)` retrieves whatever `$alter` was in the two-argument call. The point in this is that `ego()` is caller-sensitive, so that each class sees its individual `Alter::ego()` for each object. A class can initialize it to whatever it wants and use it without interference from other classes. No other class can access it (through the official interface), by definition. In the current implementation the assignment is sticky, it only works once for each class/object combination, but I don't think that's necessary. In more detail, on the first call (ever) with an object, `ego()` associates one hash with the object (through `ext magic`), which is keyed by the caller's class name to store the individual alter ego(s). Later calls from other classes continue to use it. I like to call this hash the corona because it represents the round of alternative personalities of an object, to stay with the image, and also because it is the overhead. The corona is invisible to the normal class user, but will be made internally available to support serialization through Data::Dumper and Storable. Anno	[reply] [d/l] [select]
Re^3: Beyond Inside-Out (details) by tye (Sage) on May 29, 2007 at 15:00 UTC
Something like: `sub ego { $_[0] \|\|= {}; my $corona= shift @_; my $pkg= caller(); for my $ego ( $corona->{$pkg} ) { $_= shift @_ if @_; $_ \|\|= {}; return $_; } }` [download] Yes, a much simpler and more direct solution to some of the problems that "inside-out" objects are being promoted to address¹. Note that this suffers from the usual short-comings of caller. The calling "package" is quite an ambiguous thing and calller returns only one meaning of that. I have personally written methods that have many different values for the different interpretations of "calling package" and I bet your module would fail in such situations (just like SUPER:: and its replacements fail). Consider: `package Devel::HandyMethods; sub Handy::GetOrSet { my $self= ego( shift @_ ); # ... } package My::Module::_private; *My::Module::foo= \&Handy::GetOrSet; sub My::Module::bar { my $self= ego( shift @_ ); # ... }` [download] It is unfortunate that caller provides no way to get "My::Module" when Handy::GetOrSet() gets called via `"SubclassOf::My::Module"->foo()` (assuming `@SubclassOf::My::Module::ISA= 'My::Module'`). So you should provide ways for module authors to override how you determine which package name to use. And thanks for pointing out some of the stupidity of "inside-out" objects. I still boggle a bit that such a convoluted technique for inheriting from objects that were not designed to be inherited from has become the most touted method for doing all Perl OO. Saner alternatives need more touting. (: - tye ¹ Note that "inside-out" objects were first created to address being able to subclass arbitrary objects but "inside-out" objects are now being heavily touted as the way to implement all objects and your idea certainly has a lot of advantages as far as a "best practice" way of implementing objects. But your method doesn't deal with the original "subclass arbitrary object" problem. Also, you don't address the "compile-time checking of member variable names" issue that "inside-out" objects address. Some would "use fields" as another solution. I prefer to have an array for an object and use constant subroutines for member variable names.	[reply] [d/l] [select]
Re^4: Beyond Inside-Out (details) by Anno (Deacon) on May 29, 2007 at 15:57 UTC
Re^5: Beyond Inside-Out (class) by tye (Sage) on May 29, 2007 at 17:46 UTC
Some notes below your chosen depth have not been shown here
Re: Beyond Inside-Out by xdg (Monsignor) on May 30, 2007 at 11:48 UTC
I question whether this statement is really a correct characterization of inside-out objects: # - No method of an inside-out class accesses (de-referrences) the body of its objects directly. When using inside-out classes for 'black box' inheritance, this is often exactly what is desired, e.g. subclassing an `IO::File` or other unusual base object type. Overall, it's an interesting approach and you've clearly given it a good deal of thought -- but I don't really understand how this is "beyond" inside-out objects -- rather it's an extra level of indirection and I'm not entirely clear what the benefits are or whether those offset the 'costs'. On potential costs/downsides: Compile time field name checking is lost (as someone else mentioned) Overhead of calls to `ego` (vs `refaddr`) -- Inside-out classes that offer "black box" inheritance still require overhead of calls to `refaddr`. Both can be cached outside of loops, so this is really an issue of just `ego` vs `refaddr` (which may not matter for most applications). And Object::InsideOut's array implementation with the index stored in the object itself will be faster than either approach anyway (but doesn't do direct "black box" inheritance as a consequence). With "regular" inside-out classes, getting at the lexical storage of data from outside the file scope requires fiddling with PadWalker, whereas it looks as if `ego` could be fooled because of `caller` to access underlying data. (Maybe this is covered in the XS discussion which I didn't really follow completely.) If so, "breaking" encapsulation becomes easier from the outside. (Some might consider that a feature, I suppose.) I'd be interested in a concise summary of the features/benefits. Right now I really only see one: Classes get "inside-out" property storage without needing to define fields and associated data storage hashes/arrays explicitly. (Though I personally question whether that is really a net savings in characters typed if accessors will need to be defined for most fields anyway or if some sort of field-name validation is added.) -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply] [d/l] [select]
Re^2: Beyond Inside-Out by Anno (Deacon) on May 30, 2007 at 16:27 UTC
I'd be interested in a concise summary of the features/benefits. Right now I really only see one: Classes get "inside-out" property storage without needing to define fields and associated data storage hashes/arrays explicitly. I see three major points going for the Alter proposition. Rewording the one you mention, In an Alter-class you get to keep Perl's original model, "an object is just a reference...", only each class gets to see its own reference. The ingenious twist of using a hash "inside-out", keyed by the object('s id) is not needed. If you want your object to "be" a hash keyed by field name, that's fine. If it's naturally a scalar, or code, you can have that too. Second, Alter objects get garbage collection and thread support from Perl, the class has nothing to do with that. Destructors can be freely used for other purposes. Third, though this hasn't been conclusively demonstraded in code, it is possible to provide generic hooks for Data::Dumper, Storable and the like that support all Alter classes and which can be inherited or even imported. With inside-out classes, this is hard to achieve because only the class knows where its data is stored. An Alter-object carries all its data with it, ready to dump and able to be restored. As for the cost, you have rightly observed that both techniques require the call to either `id()` or `ego()` on each access, with equal possibility of caching. The call to `id()` is essentially free, except for the call overhead, but `ego()` has a hash access in the path, which counts. Preliminary benchmarks show that the impact is measurable, but not dramatic. At the moment I am less concerned with efficiency but with getting it right. On a broader note, inside-out has shown what it takes to make black-box inheritance work in Perl. In particular, some kind of out-of-band storage is needed for which it cleverly uses the existing `refaddr()` as a primitive. My goal is to design a primitive (`ego()`) for out-of-band storage expressly for the purpose. In that sense I consider the sketch of the `Alter` module we're discussing a step beyond inside-out. This discussion (and another one on clpm) is helping me to spot potential problems and limitations of the approach. Anno	[reply] [d/l] [select]
Re^3: Beyond Inside-Out by xdg (Monsignor) on May 30, 2007 at 18:30 UTC
Alter objects get garbage collection and thread support from Perl, the class has nothing to do with that. Destructors can be freely used for other purposes. An Alter-object carries all its data with it, ready to dump and able to be restored I think these are only true if the object reference is a hash/array that stores the references for each class. If so, you give up encapsulation and the ability to do "black box" (aka "foreign") inheritance. If Alter uses inside-out techniques behind the scenes and the object reference is a key into data storage maintained by Alter with the references for each object for each class, then you still have all the garbage collection and threading problems inherent to inside-out objects. Even if you use Hash::Util::Fieldhash, that just pushes the issues into the Perl core and it's not backwards compatible before 5.9.4 (at least, I think not). -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply]
Re^4: Beyond Inside-Out by Anno (Deacon) on May 30, 2007 at 19:00 UTC
Re^5: Beyond Inside-Out by xdg (Monsignor) on May 30, 2007 at 21:04 UTC

Back to Meditations