Xiong has asked for the wisdom of the Perl Monks concerning the following question:

My current project, Devel::Comments, is nearly ready for CPAN upload. My working project folder tree looks something like this. It's the same structure I use for other projects.

# folder # possible contents ~/projects/comments/ # loose files in here are README and the like downloads/ # like original tarballs or round-trip checks file/ # might be generated or used # by a project or utility script config/ # user would have these in ~/.comments/config/ lab/ # non-text workfiles, like graphics lib/ # Perl modules under development notes/ # more or less English notes # intended to guide or track development old/ # a sort of rubble pile; next stop, trash pack/ # prebuild folder; only stuff intended for rel +ease; # mostly symlinks from other places; # and Build.PL and friends; final tarballs run/ # Perl scripts of all kinds t/ # tests, passing or failing not-t/ # bad tests, incomplete tests, dead-end ideas regress-t/ # tests converted to run under Smart::Comments util/ # nasty little hacks and demo scripts sym/ # convenience symlinks (itself a symlink) unpack/ # round-trip check of tarballs from pack/ .git/ # Git repository

Now, I would like you to imagine yourself as a fellow developer interested in pulling the GitHub repo for this project. (This actually exists at http://github.com/Xiong/devel-comments.) Perhaps you want to contribute; perhaps you want to fork; perhaps you just want to criticize. In your reply to this node, please make it clear in which role you choose to imagine yourself.

Which of these folders do you want to get in your pull?

The issue is that I put some of these folders in .gitignore; there, they are untracked and you don't see them in the repo. I really don't want to track some of them, for various reasons. I need to know how you feel, since the only point of maintaining a GitHub repo is for people other than myself. In particular, I need to know if, in certain cases where it's much better for me not to track a folder, whether it's worth the extra effort to publish it by some other route or to work around or pay the costs of tracking. Please don't say 'all' unless you really, really mean it.

2010-08-07:

Perhaps I should note, although it's a side issue, that I am not shipping the entire test suite with the CPAN upload. Many of these tests are redundant, exhaustive, or have significantly greater dependencies than are required to run the minimal test suite or DC itself.

I'm open to developing a more complex approach to the (currently unsupported) build_recommends issue. As I see it, though, it's difficult to skip a test that requires unavailable modules. I'm sure it can be done but I need to work on it.

(2):

Okay, it seems after all this time, I'm finally getting an answer to my first SoPW node: Overall Directory Organization; or at least, here's what I'm hearing: Don't worry yet about what part of your working tree to include in your repo. Change your working tree structure. Besides the replies below of moritz, FunkyMonk, and a drop-in, I have had a long, illuminating round in CB with james2vegas.

It seems that everyone builds out of their main working tree. You may skip certain things in the build but you don't have a dedicated build folder like my pack/. And MANIFEST.SKIP more or less follows .gitignore, with the former perhaps a bit more restrictive.

Most of my specific first-node questions remain unanswered by this approach to directory organization but I get the drift. This would be a fine time to go back to that node and tackle some of those specific questions. I'm all ears.

Note: Contents may change.

Replies are listed 'Best First'.
Re: What's in a Repo?
by moritz (Cardinal) on Aug 07, 2010 at 11:14 UTC

    I want to get all files that I need for building the distribution: Build.PL/Makefile.PL, lib/ and t/, maybe a README, maybe a document on hacking, maybe examples.

    If the distro needs it, additional art work, images, files for a web page.

    I don't want non-working code anywhere in the repo. I don't want any notes that nobody except that author can understand. And I want tests in t/ directly, as is custom for Perl modules.

    Perl 6 - links to (nearly) everything that is Perl 6.

      How does your list of wants differ from the final CPAN tarball?

      Again, please choose an imaginary role for yourself, to put your comments in context.

      All tests of DC are in t/ and all files in there are tests; the files in folders with similar names are things that didn't make it: bad ideas, incomplete ideas, or tests that apply to another module, for comparision.

      Note: Contents may change.
        How does your list of wants differ from the final CPAN tarball?

        I don't need generated files like MANIFEST or META.yaml.

        All tests of DC are in t/

        I had the impression they were in run/t/. Maybe I mis-read your initial post.

        Update: If it wasn't clear before: for me a release is a snapshot of a repository at a certain point, plus generated files (if necessary). As such I think that the repository structure should closely follow the structure of a CPAN distribution.

        Perl 6 - links to (nearly) everything that is Perl 6.
Re: What's in a Repo?
by jau (Hermit) on Aug 07, 2010 at 12:45 UTC
    As I see it, though, it's difficult to skip a test that requires unavailable modules.

    Not really:

    #!/usr/bin/perl use strict; use warnings 'FATAL' => 'all'; use Test::More; BEGIN { if ( eval { require Cache::Memcached } ) { plan tests => 1; } else { plan skip_all => "Cache::Memcached is not installed"; } } { require_ok( 'My::Module' ); # or whatever you want to test }

      Example: Advanced testing requires Test::More 0.94 while basic tests do not. I don't know exactly how to handle this. You're urged to get involved. I won't pretend I can't use help. But this is a side issue to the main point of this node.

      Note: Contents may change.
Re: What's in a Repo?
by jethro (Monsignor) on Aug 07, 2010 at 11:33 UTC

    I don't see different roles here. The roles of contributor and forker are the same because of the structure of git and GitHub. A critic is somewhat submitting an incomplete patch by different means if he critiques structure, code details or inner workings of your software. In that sense he is a contributor as well.

    Ultimately you are the only person in the position to know whether something in not-t or util for example is worth sharing. Only you have looked the gift-horse in the mouth

      A contributor to a project works on it and comes back with changes. A forker pulls it, does his own stuff with it, and never puts back; there is no merge. A critic doesn't make changes but yells at me in CB. You may not see it split up that way but I do. I realize the division is artificial, arbitrary, fuzzy, etc.

      All these types of involvement are welcome but if I know in which role you see yourself, it puts your remarks in context.

      Note: Contents may change.

        As far as I know the development on github mostly goes this way: If someone wants to work on your code he makes a fork of your repo and makes some modifications. Then he informs you that he has some commits there you might be interested in fetching. Also developers on githup usually check out what is done in forks of their project and fetch interesting stuff without being promted to anyway.

        What you probably meant is a real fork. That is a split in development because someone wants to move the project into a very different direction, thereby renaming it. As you might have experienced yourself with Devel::Comments, this is a step that is very seldom done and most of the time only after massive efforts to either find some lost maintainer or throw around abusive language.

        This kind of fork is something exceptional and usually the forker doesn't know in advance that he will fork eventually. So nobody will step up and tell you "I'm a forker" because you first would have to either have abandoned the project for a long time or (in his eyes) don't accept his contributions after lots and lots of discussions. So if there exists a forker like that you must know him already

        Someone who yells at you in CB is not a critic but a raving lunatic, they usually don't want any code of yours in a 10 mile radius ;-). If you welcome their contributions just add lots of bugs and failed code to your project and CB will be full of them

        To make the point in my previous post about critics somewhat clearer: There are two sorts of critics in your case:

        1) The user. He has something to critizise about the user interface or features or documentation of your module. He doesn't need any source code, he is concerned only with what you distribute and its deficiencies. We can ignore him, he has no involvement at all with your code

        2) The developer critic. He found some fault in your internals or misses a feature but knows how to add it. He may only use words but he looks at your source the same way as a contributor would, he just doesn't do the last step of really implementing the patch(es) but only tells you about it.

        And a last point: Not only are all these different roles identical in their requirements. But anyone might, after using your module, turn up to fit any of these descriptions. He might just report a bug, think about a fix and just telling you about it or really write the fix (or feature). Nobody has identified a role to you yet because nobody can see into the future.

        With git they're all the same unless you pull from them
Re: What's in a Repo?
by Anonymous Monk on Aug 07, 2010 at 11:29 UTC
    Which of these folders do you want to get in your pull?

    You mean checkout don't you :)

    How about only the (latest) files that end up in a cpan distribution? Just tag it blead (or bleaddist/bleadcpan).

    My role would be to fix a single-glaring-easy-to-fix-bug I encountered, which I would usually submit as a patch against your latest cpan tarball

    In all honestly I would never be involved with your module, because magic module is magic :)

      I mean, What contents do you want to see in that repo?

      Most people seem to think 'blead' is an alternate speling of 'bleed'. Googling 'bleaddist' turns up exactly one page: This one. Would you be willing to elaborate? I'm not asking how anything should be tagged.

      Note: Contents may change.
        What contents do you want to see in that repo?

        Everything you want to publish , as long as I can always checkout the bare bones cpan distribution branch.

        I think everything should go into your private repository, but you should only share/publish (like on github) that which isn't private. According to your original post everything should go into your public repository except maybe some config files (local passwords/keys)

        blead is blead (trunk in svn parlance, current main branch), something perlhackers will understand, and you might want to explain in some kind of REPO-README