When I read this, my first thought was delegation. Have an object that will know whether its elements are the same as another object's elements. If an element is a scalar, compare it in a set-like fashion. If it's an object, delegate to that object. And, frankly, your "object" can simply be blessing the arrayref into a class with one method - the comparator. Something like:

package Comparator;

sub new { bless $_[1], $_[0] }
sub comparator {
    # Some stuff here
}
[download]

Of course, the interesting bit is the # Some stuff here, and I'll look at implementing it later. Right now, I have interesting stuff to do and it's actually work-related! :-)