Working with public, private, and protected fields

However, I prefer using closures because I can define private and protected fields with ease.

I appreciate that you put some effort into drafting a tutorial-style meditation. However, in both this post and A quicker way to have protected and private fields?, you show a clear bias for closure-based objects over any other type. (You do realize that Objects with Private Variables was written in 2000, right?)

At the very least, for clarity, I think that you should retitle your post to something like "Closure objects with public, private, and protected fields".

Morever, I think that your piece would be much stronger if you explored several ways to do public, private and protected fields and articulated the pros and cons of each. Some people might reach different conclusions on whether the pros and cons lead to the same place.

For example, with a closure based object of the type you articulate, every private field access (from within __PACKAGE__) requires:

• Dereferencing the closure reference
• Calling the closure
• Two hash dereferences to get the access level
• Checking the access level and the caller
• Two more hash dereferences to get the value

Contrast that with an inside-out object:

package Duck::Secret;

use Class::InsideOut qw( private id register );

private secret => my %SECRET;

sub new { 
  my $self = register( bless \(my $s), shift );
  $SECRET{ id $self } = int( rand(100) );
  return $self;
}

sub guess_secret {
  my ($self, $guess) = @_;

  return $guess == $SECRET{ id $self };
}
[download]

Accessing that inside-out private field requires:

• Calling id (which is really Scalar::Util::refaddr)
• One hash lookup

So, in this example, if someone has lots of private fields and few (or no) protected fields, inside-out objects seem likely to have much better efficiency.

Adding protected accessors for inside-out objects is also easy:

# continuing previous code example

sub secret {
  my $self = shift;
  die "Secret is a protected field" if ! caller(0)->isa( __PACKAGE__ )
+;
  $SECRET{ id $self } = shift if @_;
  return $SECRET{ id $self };
}
[download]

That still has less overhead than a closure-based object.

So, maybe you'd like to consider a fuller treatement of the subject before declaring closure-based objects are the way to go.

Inside-out objects also require much less code and work. I can't see how the benefits of the blessed closure technique demonstrated in the original post make up for its lack of clarity and maintainability or its repetition.

Enforcing protection makes many nice hacks very hard, or even impossible. That's very unfortunate, in a highly dynamic language, in my opinion. I choose to trust the caller to know what the underscore means. If anyone wants to call my "private" or "protected" methods, they can go ahead. But they have been warned (by the underscore and/or (lack of) documentation).

If I could, I would vote 10 times for this great answer. I can not agree more. Juerd++.
Boris

But they have been warned (by the underscore and/or (lack of) documentation).

Unfortunately, lack of documentation doesn't guarantee a feature isn't used. :-(

--
Ytrew

Let me first say that I was once like you are, I was obsessed with trying to make Perl OO be more like other, "stricter" OO systems. I even wrote and released a module to CPAN (see Devel::StrictObjectHash) to do exactly what you are doing here (I used tied hashes though, instead of closures). I also wrote many "protected" and "private" methods which died if the wrong person touched them.

And then, after a few years of this, I realized that never once did I ever actually need any of this code (and the computational overhead that came with it). You see, I realized that I had just built in a bunch of checks, which would always return false, and never execute, because I always used my modules correctly. I then also realized that all my co-workers who used my code did the same. And I would venture to guess that any of the users of my CPAN modules, they also used them correctly. And why do I think this is so?

Because that is how I documented them, plain and simple.

I basically came to the realization, that this is a social problem, and not one to be solved with code. If my co-worker is violating my encapsulation, he/she is wrong. If they have to violate my encapsulation to make things work, then my design is wrong.

Now, from a purely code point of view, I also noticed something about your code (which was always in my code too), especially in how you deal with private and protected methods. You suggest that adding this at the begining of your code for a private method:

    caller(0) eq __PACKAGE__ || confess "setQuackBehaviour is private"
+;
[download]

    caller(0)->isa(__PACKAGE__) || confess "setQuackBehaviour is prote
+cted";
[download]

To properly implement public/private/protected methods, you need to have access to Perl's method dispatcher itself, and that is, I'm sure, not possible without patching perl itself. And to really implement this kind of behavior effectively and efficiently, you probably need some degree of program analysis and code fiddling/generation, which again means a patch to perl. And by the time you are all done with this, you might as well have just saved yourself the time and used Java/C#/C++ or some other language where you get his for free.

If my co-worker is violating my encapsulation, he/she is wrong.

Unfortunately, management rarely understands, nor cares, which party is "wrong" from the standpoint of computer science philosophy. Mostly, they just care whether or not the code you put into production breaks something; and if your patch to production code breaks something because you foolishly assumed your co-worker wasn't an idiot, you're going to have to answer for your mistaken assumption.

Never just assume competence on the part of other people, or even yourself for that matter, unless you have absolutely no choice. Double and triple check everything and verify that it is right.

Reliable systems come from multiple independent verifications and confirmation; not blind faith in the talents of the people working on them.
--
Ytrew

Unfortunately, management rarely understands, nor cares, which party is "wrong" from the standpoint of computer science philosophy.

Yes, I completely agree, but what I am talking about is not "computer science philosophy", but social contracts. Management does understand rules, and if "don't break encapsulation" is a rule, then the fault will be clear. Any team must have agreed upon ways of working, or you find that you will end up with total chaos. And this goes for industries outside of technology too, anyone who has ever worked in a resturant knows this. If you cook staff is not a well oiled and coordinated machine, which is governed by both written and unwritten rules, not only will peoples dinners be late/wrong, but there is a good chance someone could end up in the hospital by the time the night is over.

Mostly, they just care whether or not the code you put into production breaks something; and if your patch to production code breaks something because you foolishly assumed your co-worker wasn't an idiot, you're going to have to answer for your mistaken assumption.

Well if you are in a position of responsibility, you should always assume everyone else is an "idiot" unless proven otherwise. This is not to say you should walk around with a chip on your shoulder and look down your nose at them. Only that if it's your ass is on the line, you need to make sure your covered.

As for deploying/patching production and having things break, that is what staging/QA environments and regression test suites are for. If you are not using them, then you are already in deep trouble so what's a little bit more :P

Reliable systems come from multiple independent verifications and confirmation; not blind faith in the talents of the people working on them.

Of course, and through those "multiple independent verifications", someone would (I hope) spot the encapsulation violation and it would be addressed, and the programmer would recieve thirty lashes and placed in stocks to serve as a warning to others.

-stvn

how did your co-workers mistakes end up in production? seems to me you didn't double check and verify enough.

It looks like you put considerable thought into this. I applaud your hard work. Still, I have to say that whenever I encounter some arrangement that tries to protect me against myself, I get a bit defensive or annoyed or something. I feel I should be allowed to get myself into as much trouble as I want, even if it's something very dumb and avoidable. No matter how many times I had to be told not to stick my finger into the light socket, I only had to do it once to convince myself that this wasn't something I wanted to do. Well, OK, not more than a few times ;-)

I feel I should be allowed to get myself into as much trouble as I want, even if it's something very dumb and avoidable.

If the people who caused the problems were the same people who had to fix the problems, I might agree with you. :-) That's not always (often?) the case. I hate it when the person currently working on a project assumes that they will be the only person ever to work on the project. Especially when the person who ends up being asked to "just make it all work" when the person leaves the company is me. :-(
--
Ytrew

I feel your pain, and you've raised a good point. I've never worked on any sort of programmning team or project, so I've never had to clean up code, nor have mine cleaned up. I *have* come back to my own code after a few months, only to find that whatever flash of genius allowed me to create it, was no longer with me, and it was about as readable as Sanskrit ;-).

Sure it sucks to have to fix someone else's code because they didn't read the documentation, but it also sucks to be physically restrained from doing something that is perfectly logical simply because the author of an object never conceived that you'd use his object in that particular way.

Perl isn't the type of language that caters to the lowest common denominator, so in my book flexibility trumps idiot proofing.

From the TMTOWTDI dept.:

My CPAN distro, KinoSearch, is a loose port of the Java search engine library Lucene. At 70+ modules, it's a large system, so enforcing access levels is important. However, watertight access level control in Perl takes heroics, and heroics are expensive both in terms of maintenance and CPU time.

The policy I came up with (text taken from KinoSearch::Docs::DevGuide), is only a mild expansion on widespread Perl practices. It's served me well so far:

No public member variables.

Multiple classes defined within a single source-code file, e.g. TermQuery and TermWeight, may use direct access to get at each others member variables. Everybody else has to use accessor methods.

C-struct based classes such as TermInfo allow direct access to their members, but only from C (of course).

Subroutine/method access levels

There are three access levels in KinoSearch.

1. public: documented in "visible" pod.
2. private: subs which are prepended with an _underscore may only be used within the package in which they reside -- as per perlstyle guidelines -- and in only one source file.
3. distro: any sub which doesn't fall into either category above may be used anywhere within the KinoSearch distribution.

The ban on public member variables, plus some lightweight constructor argument checking, goes pretty far towards insulating KinoSearch against autovivification problems and all the rest. If there were a public API for getting at the C struct members, I'd hide them behind accessors using C function pointers, but since there's not, compile-time checking provides sufficient internal protection.

I haven't really needed protected methods. The ACL that would be handy is what they call "package" in Javaland, but I couldn't use that name for it because packages in Perl have a smaller scope, and "distro" is my imperfect substitute.

KinoSearch is currently about half/half C-struct based XS classes and hash-based classes, with a smattering of inside-out classes. I'd consider changing the internal implementation of the hash-based classes to inside-out if I thought it was important. But defining ACLs in the private API beyond what I'm doing now would be overkill, while tagging some visible methods as "protected" would just confuse a lot of Perlers.

At 70+ modules, it's a large system, so enforcing access levels is important.

In my opinion, you're drawing a conclusion based on an irrelevant metric. You either need access control or you don't. Chances are that you don't, but if you feel that you need it, don't feel that you also have to justify it; it's your code. :)

thor

The only easy day was yesterday

Thor, please elaborate on why "chances are" there's no need for access control in a system that big. Perhaps you interpreted the word "enforcing" in an absolute sense? This is open source: all access control is advisory.

Exposing a member variable in a public API has the disadvantage of freezing an implementation detail. Standard OO theory says that's probably not a good idea in either large or small systems. The distinction between large and small is in the internal API -- it's more important that components in a large system do not violate encapsulation of other components.

Say you have only two small hash-based classes in your distro and one accesses a hash member in the other directly. When you refactor and change the name of that variable, you can just check the other file and be sure that there aren't any ill consequences. In a large system, that's not feasible because it's prohibitively time-consuming and difficult to inspect all the code in the project every time you change an implementation detail. However, with the ACL policy I have in place, I can change how any hash member or leading-underscore sub behaves and feel confident that I only have to check the file I'm currently editing.

--
Marvin Humphrey
Rectangular Research ― http://www.rectangular.com

Like others in this thread I'm pretty firmly in the camp of "let the user do what they want, even if they want to do something dumb". I dont see a lot of point in enforcing privacy rules in a perl object structure. In fact, i dont see a lot of point in subroutines doing "check if I'm being used correctly" logic. And for more than one reason:

First is that such checks involve a fairly expensive overhead, second is that such checks usually just mean that instead of getting a run-time error from perl when i actually have done something wrong I get a run time error from the checker code telling me I might have done something wrong, but often with less information about what it is and why it was wrong. Lastly I find often that such checks are overly restrictive based on misunderstandings of how perl works. (The classic example is using ref to check type.)

For instance your logic for protected methods assumes that there will never need to be an interface compatible object that needs to be written that can't inherit from your base class but must work as though it does. I can work around your defense with some craftyness, but that craftyness is unnecessary if you leave off with the run-time protections and let me make my own decisions about how I use your code.

About the only time I think such logic is called for is when the action of the sub/object is "dangerous" in some way and must be used in a guarded fashion.

Many of the replies in this thread are based around the notion that if developers willfully engages in bad practices, then it is on their heads. This is fine when you don't have to deal with the consequences. I'm currently on a project where a developer keeps wanting to access the object hashes references directly. I keep explaining that this is bad practice, but he doesn't get it.

I ride his ass at code reviews, but I really don't have any authority to do more than that, and as we're all contractors, there is no one lead developer. Now you could argue that the problem is due to the project team structure, but I would argue that it also relates to the fact that blessed hash references are used to store the object attributes. Sure, any developer who violates the public/private contract is asking for trouble, but the fact is we all suffer if things start to go wrong.

I'm currently on a project where a developer keeps wanting to access the object hashes references directly. I keep explaining that this is bad practice, but he doesn't get it.

The problem with people like this is that they'll do other dumb things even if you don't allow them direct access to the hash. If they don't understand why it's a bad idea then they'll just find another "bad" way of solving the problem in my experience.