Very nice work! This is in Meditations, so I hope you don't mind some questions/feedback...

my $foo = My::Module->bloated_untestable_method;
[download]

I've definitely run into that, and although sometimes it isn't feasible (read: legacy code), bloated_untestable_method() should be re-written into underlying functionality-based subs that can be tested. I digress for the purposes of your work here.

refute_these {
    like $foo->{bar}, qr/f?o?r?m?a?t/;
    can_ok $foo->{baz}, qw(do_this do_that frobnicate);
    is_between $foo->{price}, 10, 1000, "Price is reasonable";
};
[download]

Now, I see you peeking into the innards of the object here, so it screams loudly that things should be further broken up with accessors that can be tested. This is kind of related to the above statement, and for similar reasons. Tests/assertions like this scream to me that further work needs to be done on the particulars of the object innards:

sub bar {
    return $_[0]->{bar};
}
sub baz {
    return $_[0]->{baz};
}

# or even, if attributes are related:

sub get {
    my ($self, $want) = @_;
    return $self->{$want};
}
[download]

Personally, I am one who really, *really* tries to stay as close to core as possible, unless there's a significant reason for not doing so (eg. a really good existing wheel, perhaps the one you wrote here), so I'm wondering if this distribution is meant for new code, or for aiding the writing of a test regimen for legacy/existing code.

I'll even go one step further to be a pain in your ass ;) Can you please lay out examples of the following statements for my, and future readers' purposes?

• Testing the method requires many preconditions/dependencies;
• The method includes side effects and outside world interactions that are not easily replaced with mocks;
• The method only misbehaves sporadically, doing what it should do most of the time, and the exact conditions required are not known;
• The method needs to be refactored to be properly tested, and needs test coverage to be refactored.

I promise I am not trying to be a smartass or cause you issues. I have App::RPi::EnvUI that has so many moving parts, I *do* have code within the modules that mock, test and otherwise have to perform ridiculous procedures when it understands it is in either prod mode or test mode, so I'm looking for something to make my life simpler, and would largely appreciate a more thorough understanding of how I can use your distribution (ie. here's a good example of how you can sell it).

Again, great work. Your documentation appears clean and concise (although I haven't tested it so I can't confirm its efficacy), and the code (after a very quick glance) looks nice and clean, and with comments sparsely populated where it makes sense.

Do you have a real world example to display which made you decide to write this distribution?

-stevieb

Hello stevieb,

I welcome feedback, especially the kind that mixes critique and flatter in just right proportion. However, a detailed answer to your inquiry would take some time.

The whole point of this bloated_untestable_method example was showing how Refute can help improving on legacy code (just as you point out). Refute covering your back may become a tipping point where rewriting legacy code turns feasible. In fact, my example could be worse as in:

sub big_one {
    # a lot of code here
    my ($foo, $bar, $quux);
    {
         # Big block of code nobody had courage to move into a subrout
+ine
    };

    refute_these {
        can_ok $foo, qw(so_this frobnicate), "Assigned an object to fo
+o";
        like $bar, qr/f?o?r?m?a?t?/, "bar is in right format";
        is_deeply $quux->{megahash}, $_[0]->{megahash}, "Data round-tr
+ip"; 
    };

    # even more code

}; # sub ends some 500 lines below
[download]

And adding refute_these to the mix serves both as a safety net when fiddling with code at hand and a prototype of the unit test for the upcoming replacement.

This is what I'm actually pushing at my current workplace. There's a gaping hole between the standards we set for new code and the old code's style.

For fresh code, the usage is much more subtle. I can think of several cases, but haven't tried any of those yet:

• Loading a plug-in? Run a behavior test at the spot. Make sure plugin authors are aware of the spec they develop to - you can just copy-and-paste refute block into documentation.
• Run a model round-trip test upon connecting to database. Don't wait until actual requests start failing. Roll back transaction afterwards. Will try this in my Potracheno tech debt tracker shortly.
• Writing a Foo::PP module? Create a spec-in-code so that Foo::XS, Foo::Fast, Foo::Tiny and Class::Foo developers don't have to come up with their own (slightly different) test suites.

Now all of the above looks like design-by-contract. There are some modules for that on CPAN, but it feels like a bit of overkill and a bit of "take it or leave it" for most cases.

Thanks again for your reply, and I hope to come back with some practical suggestions/examples soon.

Hello stevieb,

Having finally taken a look at App::RPi::EnvUI, I was able to come up with the following high-level thoughts:

• Testing the method requires many preconditions/dependencies;

  Your application requires modules that make no sense without a Pi. However, the UI and DB parts are pretty much decoupled from that, so I'd say one could try to resort to mock when the real thing cannot be loaded. This would simplify trying the UI out before uploading to a real thing as well as developing it.

  Design-by-Contract may be useful here to verify that the module being loaded is actually matching the expected interface. This part of Assert::Refute is far from perfect for now, so let's pretend runtime assertions are a form of contract.

• The method includes side effects and outside world interactions that are not easily replaced with mocks;

  Did turn fan off call actually turn fan off? Well, you can't tell for sure unless you see the fan with your eyes. But at least you can check with extra calls wrapped in refute_these{ ... } blocks that the system remains consistent from within. And of course there's good old if (DEBUG); or shiny new Keyword::DEVELOPMENT to minimize performance hit.

• The method only misbehaves sporadically, doing what it should do most of the time, and the exact conditions required are not known;

  Maybe not exactly this case, but how about some sanity checks for temperature sensors?

  use Assert::Refute::T::Numeric;
  
  refute_these {
      is_between( $temperature, 50, 140, "Temperature is sane" );
  };
  [download]

  At least if thermometer breaks and starts sending 0s, you notice it before the greenhouse boils.

• The method needs to be refactored to be properly tested, and needs test coverage to be refactored.

  Not applicable to specific app because it's done with tests to begin with. However, my other replies must shed some light on that particular use case.

Thanks again for your inquiry, it was quite insightful.

Came up with a couple more use cases for new code.

• Configuration sanity check - ensure that directories exist, numeric values are within bounds, sane defaults have been substituted etc after the configuration has been loaded.
• Output sanity check - ensure that links point back to the app, all needed IDs are present, credit card details are censored etc. In particular, e-mail templates have been notoriously hard to test in almost every of my workplaces.

Of course there are multiple ways to do that, Assert::Refute just provides a convenient language for both specification and report.