I got the right result with this code. I left my debug print statements in there so you can test some more. I'm not sure it gets all border cases. Interestingly the same-dir rule turned out to be a special case of the subdir rule (or not? Please test some more. Really)

sub sort_files {
        print $$a{'path'},' - ', $$b{'path'},"    : ";
        my $adir = $$a{'dir'};
        my $bdir = $$b{'dir'};;

        # b in subdir of a
        if ($bdir =~/^$adir/ && $$a{'file'}) {
                print "dir 1\n"; return 1;
        # a in subdir of b
        } elsif ($adir =~/^$bdir/ && $$b{'file'}) {
                print "dir -1\n"; return -1;
        }

        # a and b in the same dir
        #if ($$a{'dir'} eq $$b{'dir'}) {
        #        if ($$a{'file'} && !$$b{'file'}) {
        #                print "f -1\n"; return -1;
        #        } elsif (!$$a{'file'} && $$b{'file'}) {
        #                print "f 1\n"; return 1;
        #        }
        #    }
        print "   ",lc $$a{'path'} cmp lc $$b{'path'},"\n";
        return lc $$a{'path'} cmp lc $$b{'path'};
}
[download]

Note I changed your code so that $adir is simply $$a{'dir'} as it should be the same. Also put 2) before 1) and removed the condition in 2) that b should not be a file