in reply to Getting Involved with Perl 6 - 2009

Is there any plan to move the spectest suite into the Rakudo repository? Is there a reason not to? I understand it was originally part of the work on Pugs, and so it "belongs" there in that sense, but I thought Pugs was approximately dead and the spectests are the only action that repository gets anymore. Would it be a good idea to move the tests to some third repo that any implementation could use?

This has always seemed odd and clumsy to me.

  • Comment on Re: Getting Involved with Perl 6 - 2009

Replies are listed 'Best First'.
Spec test location (was: Re^2: Getting Involved with Perl 6 - 2009)
by moritz (Cardinal) on Jun 21, 2009 at 17:45 UTC
    Thanks for bringing it up, I'll try to summarize the current state:
    • The pugs repo is used as a general repository for Perl 6 related projects. Although we still call it "pugs repo", it hosts the specification, test suite, elf, mildew, STD.pm, some infrastructural code (IRC logging, evalbot, ...). We think about renaming the repository one day.
    • The test suite won't move into the rakudo repository. It is meant to be implementation agnostic, and moving it into the repo of an implementation would be counter-productive (and it's used by other programs as well, namely STD.pm and elf). Maybe I'll move it out into a completely separate repository one day.

    Currently none of these issues are pressing, so I won't hurry. Especially I want to sort out the rest of the old pugs test suite before doing any kind of move. Since the easy-to-verify tests have already been moved, this is tedious work, and going to take some time. I also consider using git instead of SVN, but I'll have to wrap TimToady's head around git first ;-)

    Any move will be carefully coordinated with the heads of the others projects, of course.

[reply]

      Thanks for your reply! I hope you don't mind if I have a few more questions.

      I find git to be very SVN-like when using Cogito. In fact, I rarely type 'git' at all. (I guess that's more a note for TimToady than you.)

      What's involved in sorting out the old Pugs test suite? Off the top of my head, I'd think that it means: 

      foreach my $file ( @pugs_test_files ) {
    my $new_file = reasonable_spectest_location_for( $file );
    move $file, $new_file;
    add_fudges( to => $new_file, for => $_ )
        for @perl6_implementations;
}

      [download]

      If I were doing this, I think the hard parts would be,

      1. Figuring out where to put the new file in the spectest directory, since I'm not very familiar with the spec. (Perhaps this is a way to get familiar.)
      2. Fudging anything other than Rakudo or Pugs, since I don't have them.

      After that, I'm guessing it's grunt work of running the tests and seeing how they blow up to get them fudged properly for Rakudo. (Skip tests that die, and 'todo' tests that fail.) If I were doing it, I'd probably let the other implementation authors be responsible for their own fudges. Or is that rude?

      Is there more involved than that?

[reply]
[d/l]
        The most difficult problem while moving the tests is deciding if they still conform to the current spec.

        Some tests use (or test) rather obscure features of which I don't know if they are specced, and if yes, where. The specs are a massive lump of text (more than 31k lines) and rather dense, so that's not always easy to decide. Try to search for the operator but in English text isn't entirely easy, for example.

        OTOH fudging isn't really necessary for moving tests into spec/: all implementations that we care about don't expect all test files to pass, only a selection of them. (Pugs does, but since pugs isn't maintained anyway, I don't care for now).

[reply]
[d/l]
[select]

In Section Meditations