in reply to I need your help with parsing TAP output

After this came up in the CB, and jkva was kind enough to point to this thread, I saw it was basically unanswered, despite being front-paged. Looking at this, I have a couple of questions.

  1. What is wrong with Test::Harness that needed fixing? I'm not saying that there isn't anything, only that I don't have the experience with Test::Harness that has shown much that I wanted to do that wasn't available. Except have it figure out my plan for me - counting isn't fun.
  2. What is it exactly that you are asking of other monks? Are you asking us to run tprove with our existing tests? Is there any rewriting that we might have to do, or is this completely compatable with Test::More and its ilk? It's not entirely clear to me what you're asking of us, and maybe others wondered the same thing.
Thanks,

  • Comment on Re: I need your help with parsing TAP output

Replies are listed 'Best First'.
Re^2: This is what's wrong with Test::Harness
by Ovid (Cardinal) on Jul 30, 2006 at 07:23 UTC

    Thanks for the feedback. I'll tackle the second question first: I was hoping monks would simple run the tprove program against some test suites let me know if there were any parse errors. Fortunately, I have a newer version of tprove which should be out soon and will make this easier to do. No coding's required, but it anyone wanted to, I wouldn't complain!

    As for the problems with Test::Harness, those are documented here. Even the creators and maintainers of Test::Harness agree that it has to go. Basically, we're talking about code that has evolved over almost 20 years into a big ball of mud. This has led to confused responsibilities within the systems and made it very hard to extend it to add new features.

    What do you want to do with that test output? Want colored tests? Want to only report failures? Want to throw them up on a GUI like jUnit and friends? Want to transform them into something your language's test harness can understand? Want them as XML docs? Want to email them? Want to have them run as your wallpaper? Forget it. It turns out that you can do those things, but it's very hard to get right. Attempts to do those things invariably turn out to be buggy and this is because Test::Harness just can't do what we want but simply refactoring Test::Harness becomes difficult because so much code depends upon its globals or other quirks. Heck, if you don't believe me, just look at the bug reports and try to fix a few of those :)

    Cheers,
    Ovid

    New address of my CGI Course.

[reply]

      Hi Ovid. There are a few implementations of TAP out there in C (such as libtap). How about creating a backend to one using XS?

[reply]

        I'm not trying to generate TAP, I'm trying to parse it and make a test harness available for it. That being said, I hope to eventually include a TAPx::Parser::Source::Executable or something similar which will produce a TAP stream based on the output of libtap or similar things.

        Cheers,
        Ovid

        New address of my CGI Course.

[reply]

In Section Meditations