This got rather long. Testimony to the thoughtfulness of your use cases; thank you again.

Here are some of the things I've used run-time reflection of some kind for, in various languages:

• When how to perform an operation depends on the type of more than one thing. For example, if you have a Shape class and want to implement Shape::intersection to compute the intersection of two shapes. Dispatching based on one of the shapes is easy; $shape1->intersection($shape2) will find the type of $shape1 and dispatch appropriately. But unless the other shape's type is known at compile-time, there will need to be some kind of run-time type information available to know how to compute the intersection. If $shape2 is coming from a database or the network, its type certainly won't be available at compile-time.

  (Starting with the bit I've emboldened)Sorry, but the second half of that sentence is obscuring the situation.

  You will have to know what type of shape is is, in order to instantiate the object. True regardless of whether you have:

  • A generic Shape class that (say) represents all shapes as lists of vertices. In which case you would call a single constructor for all shapes coming from the DB or network will be of type Shape and that will be known at compile time.
  • Specialised (sub) classes for each type of shape (Rect Circle Polygon). In this case, something within the data input will identify what shape this set of data represents, and you will then you will call the particular constructor for that shape. And all of those constructors will be known at compile time.

  This is not introspection. Because you cannot introspect an object until it exists. And you cannot construct it into existence if you need to introspect it to do so.

  This is simply data-driven code. You read the data, inspect a field with that data, and then dispatch to a constructor based upon what you see there. This can be done in good ol' introspection-less C something like:

  char * buffer = malloc( ... );
  read( source, buffer );
  int type = buffer[ 0 ];
  
  void *o;
  switch( type ) {
      case RECT:    o = makeRectFromString( buffer ); break;
      case ELLIPSE: o = makeEllipseFromString( buffer ); break;
      ...
  }
  [download]

  And if you can do it this way in C, you can do it in any other language, including those that support reflection.

  For your intersection problem, any language that supports method overloading, C++, Java etc., will allow you to code methods within your Rect subclass with signatures of:

  class Rect;
  bool intersect( Rect* );
  bool intersect( Ellipse* );
  bool intersect( Polygon* );
  ...
  [download]

  So that invoking someRect->intersect( someShape ); will get to the right code without introspection.

  In Perl, which doesn't support method overloading, you would have to dispatch internally to the (single)intersect method, but that's a Perl OO-model limitation.

  ---

• Loading things at run-time. For example, I wrote a streaming database system in Java that started by reading a configuration file, then loaded the classes named there. It was best to look at the class as soon as it was loaded and make sure it was a type I could deal with, so more information was available for error reporting.

  This is the 'plug-in' scenario. Loading a class at run-time from a filename read at run-time. This part is data-driven. It cannot be reflection, since there is nothing in memory upon which to reflect.

  For the "make sure it was a type I could deal with" part of the equation, if all plug-ins are derived from a base class, then the only check required is to verify that the class loaded is derived from that base class.

  This can be done through Java.Lang.Class or Java.Beans.Introspector. But, as I found in several sources whilst refreshing my latent memories of these, there are costs:

  The Costs of Usage

  Reflection and Introspection are powerful tools that contribute to the flexibility provided by the Java language. However, these APIs should be used only as needed and after taking into account the costs associated with their usage:

  • Reflection and Introspection method calls have a substantial performance overhead.
  • Using reflection makes the code much more complex and harder to understand than using direct method calls.
  • Errors in method invocation are discovered at runtime instead of being caught by the compiler.
  • The code becomes type-unsafe.

  What's the alternative? try{ ... } catch{ ... }

  Once the class is loaded, try instantiating a (minimal) instance, and exercising the required methods. Catch and report any errors. You can even check that the loaded class methods return sensible values--And that's something that no amount of reflection can do. All your validation is performed immediately after loading.

  The rest of your main application can be written against the plug-in base class. As you can pass or use an instance of a derived class anywhere you can use an instance of its base class, your main application will written in terms of direct method calls, and be compile-time type-checked. No further need for run-time type checks or try/catching.

  You simply define all your main application method calls as taking instances of the base class, and pass instances of the run-time loaded derived plug-in class.

  The advantages are all the opposites of the costs listed above. And, as you do not need to use reflection, you can use this technique in any language that supports runtime loading of classes and exception handling. Eg. Perl 5.

  ---

• Working around quirky behavior. I may know that a particular class will handle a particular situation incorrectly, so I can detect that class at runtime and try to work around the problem. For example, if you know a particular class has trouble with unicode, maybe you strip it out before sending it to that class.

  I think what you are saying here is that you can use reflection to construct a workaround to the quirk, not detect the need for it. If you know enough to write code to use the reflection APIs, you know enough to write a non-reflective solution--if that is possible. The question then becomes, is a non-reflective solution possible, and that will depend very much on what the quirk is, and what language your using.

  But the classic solution to this is to construct a subclass that inherits from the quirky class and override the troublesome methods. For your unicode-unaware example, you convert from unicode on the entry to the subclass methods, call the superclass method, and convert any returned strings back to unicode on the way out.

• Displaying information. I have a program that manages several different types of objects. I just added on a GUI displayer, and the easiest way to do that was to write a little display module, and have it say "if this is type 1, display it this way; if it's type 2, display it this way; etc...". It could have been done through sub-classing, but it would have been much more complicated.

• Automatically providing information for things like GUI editors and SOAP interface generators. It is convenient for these systems to be able to walk a class's methods to see what it can do, to make that functionality available over a network connection or a GUI.

  If you code each method with a (Java-style) toString() method, this problem is 'solved'.

  Yes, I know that doesn't solve the problem for third party classes that either fail to define a toString() method, or define one and return "Verboten!Keep your nose out sucker!", or worse :)

  More seriously, if I supply symbol files for libraries to my Symbolic debugger, then if can decode and display the contents of structs and the like. Similarly, the built-in GUI editor in the only IDE I ever spent any serious time with (PWB), utilised those same symbol files plus other compiler generated files to do some pretty remarkable things including single stepping the code backward as well as forward.

  But I will admit that these are both good use cases for reflection. The questions it leaves me with are:

  1. Is it necessary or desirable to load up every class, of every application, with all the overhead that reflection entails, in order to meet these use cases?
  2. Or would it be better to follow the PWB practice of placing this information in an ancillary file during compilation, and only loading it for those (few) use cases that need it?

     Or in the case of a dynamic language where compile-time is the early stages of run-time, have a run-time switch on the interpreter that enables reflection? Or perhaps a per-class or per-module pragma that enables it?

(My) conclusions

1. I have no doubt that all of these use cases could be met using a language (like C) that has no introspection capabilities, if the need was strong enough. It might entail using compiler generated ancillary files, or in effect, constructing a limited reflection capability.

2. As I think I've shown, three of the five above can not only be met using standard OO mechanisms, I would say that those three would be better met that way.

3. The other two, (grouped together and shown last), are strong use cases for reflection, where that capability is available.

   Though whether they are common enough to warrent the inclusion of introspection in a language is open to debate. If the reflection is done, as in Java, by (as I understand it), pulling apart the bytecode at run-time, the costs when it is not used are negligable.

   But in languages or frameworks where introspection must be support through the retention of compile-time parsing tables or the construction of code tree decorations, the costs in terms of space and time can be significant enough that they should only be done on demand, through the use of compile-time switches or pragmas, not by default.

4. In general, I think that just as inheritance was overused and abused in the poineering days OO, and still is by novice users, so introspection is equally easily abused. And 5 or 10 years from now, it will probably have gained as bad a rep. when people step back from its rising popularity on the basis of RoR and resurgent interest in Objective-C.

   Like any technology, used sparingly with understanding of the costs, it will serve some use cases in ways that nothing else can.

   But if it is overused, just because it can, to solve at run-time use cases that are better served by compile-time solutions, I see it suffering a backlash as the mainenance costs become evident.

I hope that if anyone read this far, they will find some of this as useful as I have.