Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re^3: Testing my tests

by LanX (Saint)
on Feb 27, 2017 at 02:54 UTC ( [id://1182924]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Testing my tests
in thread Testing my tests (mutation testing)

> We all do this, by default, don't we?

manually yes, not generated.

Szabgab only hinted about tests that handle code like $a+ switched to $a-

so in the following case

sub foo { my ($a,$b) = @_; return $a + $b; }

testing foo(*,0) (with * any number) will always pass, even if you change + to -

If several pairs of values were generated by a random generator and the result was approved manually, then the likelyhood of a false positive would be minimal.

And new test input could be generated quickly, depending on the manual decision if the code change was really intended or just a bug.

As I said to (semi-) automatize this one needed to parse already a lot of the functions body.

Anyway I don't want to elaborate more, without further explanation by the OP. (the requirements sound paradoxial)

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Je suis Charlie!

Replies are listed 'Best First'.
Re^4: Testing my tests
by szabgab (Priest) on Feb 27, 2017 at 05:39 UTC
    I am not sure why do you see the requirements paradoxical. You were the closest by far to what I was thinking and brought an excellent example. I've updated the original post with that and some further explanation.

    And you are right this probably requires the full introspection of all the code.

      > the requirements paradoxical.

      because without further explanation it sounded like

      • wanting tests that never fail
      • or having test heuristics intelligent enough to make you unemployed.

      at least now I'm sure what your up to.

      Regarding your update, you can't avoid false positive tests.

      But like I said you can reduce the likelyhood of them drastically for a reasonable prize by bundling many random test vectors.

      Some my produce false positives, but hardly all of them.

      In our case (2,0) may be OK but (666,42) would not.

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Je suis Charlie!

Re^4: Testing my tests
by stevieb (Canon) on Feb 27, 2017 at 03:29 UTC
    "(the requirements sound paradoxial)"

    They do indeed, and with the respect I have for the OP, that's why I've been throwing out such odd concepts all day.

    I don't know what exactly OP is looking for. Gabor didn't come back with further insight, so trial by current knowledge or at best peer-review is all we could throw forward.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1182924]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (4)
As of 2024-04-21 03:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found