Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^4: Testing methodology, best practices and a pig in a hut.

by BrowserUk (Patriarch)
on Feb 27, 2008 at 19:28 UTC ( [id://670715]=note: print w/replies, xml ) Need Help??


in reply to Re^3: Testing methodology, best practices and a pig in a hut.
in thread Testing methodology, best practices and a pig in a hut.

but complaining that a certain unnamed hypothetical test suite seems big to you is just as silly as complaining that you used string eval in a certain unnamed hypothetical production program.

My point was that test suites constitute an important part of a production development, even though they may not run on production machines. They consitute a significant part of the development effort. But the prevelant idiom, the use of Test::*, applies different standards to the two sides of projects.

  1. String eval is the underlying basis of what the Test::* modules do.
  2. If you encountered the level of c&p coding that exist in most Test::* test suites, in production code, you would (rightly) be up in arms about it.
  3. If you encountered a (say) config file validation scheme in production code that did:
    warn 'Line 1 of config file invalid' if $line[ 0 ] ne '[Section 1]'; warn 'Line 2 of config file invalid' if $line[ 1 ] =~ m[^filename:(\S+ +)] and -e $1; warn ....

    For 400 lines, you'd probably question the methodology used.

That doesn't mean I want to throw a novice head-first into the whole debate...

How many novices have responded in this thread? Where else should such debate take place? Or perhaps you feel that no debate on this subject is called for?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^5: Testing methodology, best practices and a pig in a hut.
by chromatic (Archbishop) on Feb 27, 2008 at 20:57 UTC
    String eval is the underlying basis of what the Test::* modules do.

    You're going to have to show some code to prove that.

    If you encountered the level of c&p coding that exist in most Test::* test suites, in production code, you would (rightly) be up in arms about it.

    Even if I thought you had looked at "most Test::* test suites", the existence of bad code does not prove that it's impossible to write good code. Remember, I extracted Test::Builder so that people could write their own Test::* modules in part to reduce duplication in their test suites. I've also argued for years that refactoring tests is just as important as refactoring production code.

    How many novices have responded in this thread?

    Your original post talked about people who were learning through doing and implied strongly that you think a contrarian position is important for people who haven't been knocked around enough to see the nuance in things.

    If that's not novices, then perhaps you could rephrase things to be more clear. (Several other people in this thread have asked "What does this have to do with the Test::* modules?")

    Or perhaps you feel that no debate on this subject is called for?

    I suspect that other people object to what you write not because you're wrong but because you can be so unpleasant about it.

      You're going to have to show some code to prove that.

      From Test::Builder v0.78. The bit between big black lines looks an aweful lot like a string eval to me?

      sub cmp_ok { my($self, $got, $type, $expect, $name) = @_; # Treat overloaded objects as numbers if we're asked to do a # numeric comparison. my $unoverload = $numeric_cmps{$type} ? '_unoverload_num' : '_unoverload_str'; $self->$unoverload(\$got, \$expect); my $test; { local($@,$!,$SIG{__DIE__}); # isolate eval my $code = $self->_caller_context; # Yes, it has to look like this or 5.4.5 won't see the #line # directive. # Don't ask me, man, I just work here. ###################################################################### +########## $test = eval " $code" . "\$got $type \$expect;"; ###################################################################### +########## } local $Level = $Level + 1; my $ok = $self->ok($test, $name); unless( $ok ) { if( $type =~ /^(eq|==)$/ ) { $self->_is_diag($got, $type, $expect); } else { $self->_cmp_diag($got, $type, $expect); } } return $ok; }

      And cmp_ok() is used as the final step of

      1. is_eq
      2. is_num()
      3. _is_diag()
      4. isnt_eq()
      5. isnt_num()

      Same file as above. String eval between the thick marks:

      sub _regex_ok { my($self, $this, $regex, $cmp, $name) = @_; my $ok = 0; my $usable_regex = $self->maybe_regex($regex); unless (defined $usable_regex) { $ok = $self->ok( 0, $name ); $self->diag(" '$regex' doesn't look much like a regex to me +."); return $ok; } { my $test; my $code = $self->_caller_context; local($@, $!, $SIG{__DIE__}); # isolate eval # Yes, it has to look like this or 5.4.5 won't see the #line # directive. # Don't ask me, man, I just work here. ###################################################################### +###### $test = eval " $code" . q{$test = $this =~ /$usable_regex/ ? 1 : 0}; ###################################################################### +###### $test = !$test if $cmp eq '!~'; local $Level = $Level + 1; $ok = $self->ok( $test, $name ); } unless( $ok ) { $this = defined $this ? "'$this'" : 'undef'; my $match = $cmp eq '=~' ? "doesn't match" : "matches"; local $Level = $Level + 1; $self->diag(sprintf <<DIAGNOSTIC, $this, $match, $regex); %s %13s '%s' DIAGNOSTIC } return $ok; }

      And that function underlies the following api's:

      1. like()
      2. unlike()

      Proof enough?

      You want rude. This is what I really think of Test::*

      Reimplementing a small subset of perl's native functionality--string/numeric comparisons and regex--, using a large, slow, verbose and complex hierarchy of OO code modules, in order to compare scalar variables against constants, when at the final step you are just going to use string eval and let Perl do those comparisons anyway, just so that you can convert the boolean results into a stream of 'ok's and 'not ok's.

      And that, just so that you can tie up stdin & stderr in order to produce a set of useless statistic is, in a word, fascile(*).

      (*)facile: arrived at without due care or effort; lacking depth;


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        I suppose "I don't like modules that use string eval" is a fine objection if you want it to be, but I was under the impression that your meditation here was an attempt to argue that blindly applying conventional wisdom as prohibitions without understanding why it's conventional wisdom and why the real world can be more nuanced than it is is a bad thing.

        So color me somewhat confused.

        The bit between big black lines looks an aweful lot like a string eval to me?

        Yep, and that's one way to do it. My Perl 6 reimplementation doesn't use string eval at all, for example. It's unnecessary in Perl 6. I believe my original Perl 5 T::B code dispatched to very small subroutine references. As it turns out, string eval happens to be faster and clearer in Perl 5 than maintaining a comprehensive dispatch table of all of the possible comparison operations. If there's a simpler, faster, and clearer way to do this, I'm sure Schwern would take a patch.

        Similarly, I believe that like() uses string eval so that the code will work on truly ancient versions of Perl 5 that don't have qr//. I don't support versions of Perl older than 5.6.2, but Schwern's willing to do so here. If you have another solution that meets his goals, I'm sure he'd take that patch too.

        However you said "String eval is the underlying basis of what the Test::* modules do." The code you quoted is not proof of that. How can it be? cmp_ok is not the underlying basis of what the Test::* modules do. It's not the underlying basis of Test::More. It's not the underlying basis of Test::Builder. Thus it's not the underlying basis of any Test::* module.

        (What's the underlying basis of T::B? ok() and, to a lesser extent, diag().)

        Reimplementing a small subset of perl's native functionality--string/numeric comparisons and regex--, using a large, slow, verbose and complex hierarchy of OO code modules, in order to compare scalar variables against constants, when at the final step you are just going to use string eval and let Perl do those comparisons anyway, just so that you can convert the boolean results into a stream of 'ok's and 'not ok's.

        My goodness, you're right. It's almost as if we wanted to test Perl code with Perl code. It's almost as if the code in T::B exists to report the locations of failures correctly, handle overloading properly, manage threading appropriately, and provide a uniform interface available from Perl 5.004 or earlier to the latest bleadperl, while allowing hundreds of testing modules to interoperate smoothly in the same process without having to know the internals of even those modules not yet written and without requiring everyone who wants to write tests in Perl to reinvent all of that badly and discover and, hopefully, fix the same weird bugs that everyone who writes testing modules in Perl would have had to fix eventually.

        The horror.

        fascile

        I assume you meant facile, which means "easy", "unconstrained", and "pleasant". Thank you. That's why we wrote those modules.

        (If you meant "superficial", I'd love to see the code you have which solves the same problems. Denying that many people have these problems and have solved them very satisfyingly is not the same thing.)

        As always, you may have the last word in this discussion.

      I suspect that other people object to what you write not because you're wrong but because you can be so unpleasant about it.

      Hm. I see the debate is ended. Contrast this question:

      Or perhaps you feel that no debate on this subject is called for?

      With this statement:

      I don't see the value in being contrarian solely for the sake of making the world less clear.

      And now think about pots and kettles.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        'I see the debate is ended.'

        And you lost.
        'I'd love to see the code you have which solves the same problems. '

        Yes, did you ever do that BrowserUk? It is fun to watch you get put in your place, but it would be nice to know that you can put your money where your big mouth is.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://670715]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (3)
As of 2024-03-29 01:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found