ichimunki has asked for the wisdom of the Perl Monks concerning the following question:

Too many questions, I suppose-- and I've looked around the major sources for guidance on these, but:

Is there a "standard" Perl development directory tree that people find useful or normal?
like /project /project/lib /project/etc /project/*. I'm guessing this is largely personal style, but on larger projects is there any usual arrangement?

Since the language is not compiled into a static binary, is there any reason for an application to use something like make to build itself? Or would simply zipping up the directory tree be sufficient?

  • Comment on Development Directory Structure and Technique

Replies are listed 'Best First'.
Re: Development Directory Structure and Technique
by Fastolfe (Vicar) on Jan 03, 2001 at 19:30 UTC
    On our systems, we use an /appl heirarchy for major projects. Thus, with a well-defined project might have some or all of these:
    /appl/project/bin main program stuff for 'project' /appl/project/etc configuration /appl/project/var state information /appl/project/tmp temporary files (maybe use /tmp?) /appl/project/lib libraries, modules, etc. /appl/project/doc developer and minimal usage docs
    Not all of these are particularly necessary (especially if the project is small, in which case none of them necessarily are). Sometimes this makes module installation more straightforward if they have specific needs and don't necessarily want the module installed publicly or getting the SA to do it is too much trouble or time. Just tell CPAN that the prefix is /appl/project, and use lib '/appl/project/lib';
Re: Development Directory Structure and Technique
by coreolyn (Parson) on Jan 03, 2001 at 18:00 UTC

    I'm looking forward to seeing answers to this one myself, as I've experimented with several styles/structures and I'm not 100% content with any of them. In the spirit of sharing, here's the structure I've evolved:

    #root (primary executable scripts) apache_root/projectname # project maintenance scripts apache_root/projectname/scripts<sup>1</sup> # modules apache_root/projectname/lib # Archives (place where maintenance scripts # pput backup .tgz's) apache_root/projectname/dist

    Looking at projects via CVS is a good way to see how other's do it, though I do not see any standard pattern accross various projects emerging.

    coreolyn


    1 I reccommend NEVER to use a 'bin' directory in your projects lest a hasty script or typo waste your systems /bin (Not that I've learned that the hard way ;)

Re: Development Directory Structure and Technique
by wardk (Deacon) on Jan 03, 2001 at 21:49 UTC

    I like both Coreolyn and Fastolfe's suggestions. With one comment on the apache_root/projectname/lib approach.

    In the case of a CGI project, I would keep any project modules in a directory outside the apache_root structure ( as well as any conf directories) as a misconfiguration by the webmaster can expose those to the web, you won't want to have someone inadvertently get an index view of your libs/conf directory where you may have stashed some important database connection information.

      I second that - it is a security risk.

      I do like both Coreolyn & Fastolfe's ideas. I am trying to organize my projects better these days but so far I keep changing my mind each time I start a new one. At least it's better than when I used to dump EVERYTHING in one folder.

      My current directory plan is a bit like this:
      /work/project-a/database - store databases in here /work/project-a/libs - modules, libraries /work/project-a/temp /work/project-a/backups - archives of previous working versions /work/project-a/webroot - Web folder starts here /work/project-a/utils - command line utils I may need /work/snippets - useful code snippets to save and reuse.
      I like Fastolfe's use of a docs folder. I really, really, really should do that. Problem is, I work in a company where I'm the only guy who can 'use Perl;' - everyone else uses VB\VBScript. Maybe that's a reason TO document more?

      Update: To answer ichimunki's second question, (if I understand it correctly), so long as you don't rely on anything above your project root and use relative links, then you should be able to zip up the project folder when you are ready to move from development to production, or whatever.
Re: Development Directory Structure and Technique
by dws (Chancellor) on Jan 03, 2001 at 23:54 UTC
    Lately I've been taking a cue from Extreme Programming, and am writing tests cases for the functionality I'm about to implement. (The goal is to write test cases before code, but sometimes this isn't practical, and I have to grow the test cases along with the code.) I have a simple test harness that I'll write up later, and a number of test scripts or inputs, and "blesses" test outputs. (The harness generates outputs, and compares them with their "blessed" counterparts.) These all have to live somewhere, so I keep an additional directory in the hierarchy for them:
    /appl/project/tests/
    
      Excellant, this is something which I really should start to do. I just haven't wanted to learn how to use/write a framework for carrying out testing.

      If you could upload your harness that would be excellant. Just had a look on CPAN and there appear to be some testing frameworks there as well. I think I should start playing around with some. (Trying searching CPAN for test.)

      Cheers!

        I'm planning on doing a writeup on my test harness, and on making the code available. It just needs a teeny bit of cleanup to get it up to Monastery standards.
Re: Development Directory Structure and Technique
by beppu (Hermit) on Jan 04, 2001 at 15:01 UTC

    When I code in perl, I try to keep CPAN in mind. I'm writing a module called Embedix::DB and the directory structure looks like this:

        .
        |-- Changes
        |-- DB
        |   |-- CML2.pm
        |   |-- ECD.pm
        |   `-- Pg.pm
        |-- DB.pm
        |-- MANIFEST
        |-- MANIFEST.SKIP
        |-- Makefile.PL
        |-- README
        |-- bin
        |   `-- ebx
        |-- htdocs
        |   |-- backend_api.pod
        |   `-- index.html
        |-- sql
        |   |-- pg_reset.sql
        |   `-- pg_schema.sql
        `-- t
            |-- 00_pg.t
            `-- data
                `-- textutils.ecd
    

    This is a structure that's similar to a lot of what you'll find in CPAN, and I find that it works pretty well.

    The module installation framework provided by the modules in the ExtUtils::* namespace is a huge advantage Perl has over other languages. I think it's awesome that practically every module in CPAN can be installed by doing:

        perl Makefile.PL
        make
        make test
        make install
    

    That kicks ass. You can write your Makefile.PL such that module dependencies are checked for. You can also have make install put executable scripts in /usr/bin (or whatever $PREFIX/bin happens to be for your perl installation). As an example, here's my Makefile.PL for Embedix::DB:

        use ExtUtils::MakeMaker;
    
        WriteMakefile(
            'NAME'          => 'Embedix::DB',
            'VERSION_FROM'  => 'DB.pm',
            'ABSTRACT_FROM' => 'DB.pm',
            'EXE_FILES'     =>  qw(bin/ebx) ,
            'PREREQ_PM'     => {
                'Embedix::ECD'      => 0,
                'DBI'               => 0,
                'Pod::Usage'        => 0,
                'Test'              => 0,
            },
        );
    

    PREREQ_PM is for specifying module dependencies, and EXE_FILES is for specifying a list of scripts to install. EXE_FILES doesn't work too well for webapps, but you're pretty much on your own, when it comes to installing those, because everyone installs Apache differently.

    You know what else? I love having a standardized testing framework, too. I've even started to like writing tests for my modules, because it's kind of fun to run "make test" and watch it succeed or see how much time it takes. I'm really surprised the Python or Ruby people haven't tried to copy this aspect of Perl, yet. It would really help them out.

    Well, that's how I develop. This style works for me.

Re: Development Directory Structure and Technique
by jplindstrom (Monsignor) on Jan 04, 2001 at 07:37 UTC
    This is a very interesting question, and I too would very much like to hear about other people's experiences.

    This is how I set up one project. The directory structure seemed to work fine and has now become a kind of template when I start new projects. The basic structure looks like this:

    PlayerReport
    PlayerReport\Project Management
    PlayerReport\Modules
    PlayerReport\Release
    PlayerReport\lib
    PlayerReport\man

    The end product is a application used to parse log files from online casino games (like Blackjack, Craps etc) and create reports from them.

    PlayerReport
    The name of the project

    PlayerReport\Project Management
    Various documents regarding external documentation, formats, planning, schedules etc. Word-, Excel-, text files.

    PlayerReport\Notes
    Misc notes and files that could be nice to keep but doesn't belong anywhere else in the project anymore, e.g. obsoleted source files.

    PlayerReport\Modules
    All external modules that needs to be installed for the application to be run/installed.

    PlayerReport\Release
    Two things: First, the entire application ready to be installed in another environment (production or test). In my case this means a ZIP file with a standalone .exe version of the application created with perlapp, along with all other external files like config files, data files, images etc. When/if this application will be ported to Unix, this package will be the main thing to change.

    Second, a script that makes a "build" of the application, performing regression tests (see below), creating the standalone .exe-file, and creating the distribution ZIP file.

    PlayerReport\lib
    This is where the actual code lives. In this directory I have a .pl file for the application. Also, all files needed to run the script, like config-files, small test-scripts for trying things out, etc. Modules live in subdirs:

    PlayerReport\lib\GameLog
    PlayerReport\lib\GameLog\Userlog
    PlayerReport\lib\GameLog\Session
    PlayerReport\lib\GameLog\Round
    PlayerReport\lib\Game
    PlayerReport\lib\Game\Hand
    PlayerReport\lib\Game\Bet
    PlayerReport\lib\Game\Decoder
    PlayerReport\lib\Game\Card
    PlayerReport\lib\Game\Reels
    PlayerReport\lib\GameView

    PlayerReport\lib\t
    The test code for the most important, most coupled, most volatile classes. I still don't perform regression tests on all classes :(

    I perform regression tests both on a "holistic" level, feeding the entire system certain input, excpecting certain output, and on a unit level.

    PlayerReport\lib\t\Holistic
    PlayerReport\lib\t\Game
    PlayerReport\lib\t\GameView
    PlayerReport\lib\t\GameLog
    PlayerReport\lib\t\...

    PlayerReport\man
    The automatically generated man pages for all classes. I have written a tool that "flattens" class documentation, bringing the POD of all parent classes into one HTML page so I have the entire interface for a class in one file.

    PlayerReport\man\GameView
    PlayerReport\man\...

    ...

    /J