>> Isn't cpan a wrapper for MakeMaker...

Sure. I've got Module::Build created modules which can do a whole bunch of things, but are at the single module level. You can build, test, and install. The equivalent of "make" built C tarballs in the unix world.

The problem comes at the next level, which have to do with module/dependency and repository. You want to do things like update a server with a single command, based on, say a bundle (and have the package system do the minimal work). MakeMaker and Module::Build are below this level. So we're talking about something that fulfills the role that yum/rpm and apt-get/debian pkgs, does. It's a dependency/repository engine that may be used from a UI or programmatically.

Since cpanp is the newest, and seems to be the anointed replacement to CPAN.pm, it seemed logical to try to master the UI first as this is the driver for the underlying dependency/repository engine. It's a fast way to play around with the underlying technology it is built on. If I can't get that working, then I'd be a little hesitant about the technology (not the Build system, but the dependency/repository system). I'm pretty confident I could do something on Build as this seems solid. The question is whether I'd have to write my own "rpm like tool". This seems absurd considering cpanp's existence.

Note I'd used cpan a lot, but just to install things, and so I was familiar with how fairly robust that was, and at least new the places where things would occasionally break. It was this experience that gave me confidence this would be a "piece of cake."

Plus this is such a basic problem for module systems that I would assume this is a completely solved problem (considering how well cpan has worked).

Note, I don't know a lot about MakeMaker, but assume Module::Build is equivalent.

CPAN.pm and CPANPLUS.pm both have features for automatically following dependencies -- that's the main advantage over manually typing the build commands.

Also, they both support Module::Build based packages, as well as the older ExtUtils::MakeMaker style.

When we finally get to the stage where nearly everyone is using CPAN/CPANPLUS (or some similar system of automatically dealing with dependencies, e.g. Debian's debs via apt-get), then life will get much easier for CPAN authors. At present there's a direct trade-off between modularity and ease-of-adoption, because if your module depends on a dozen other modules, a large percentage of the perl users out there will give up on the "download & install" grind before finishing.

Update: Caveat: Forgot about Module::Install, which is a nice hack to alleviate this problem.

>> Also, they both support Module::Build based packages, as well as the older ExtUtils::MakeMaker style.

I'm using the Module::Build approach which seems to do a decent job of the basics. However, at the module level I would expect some things were a little easier. Like installing files to a custom place, like /var/www/html.

>> When we finally get to the stage where nearly everyone is using CPAN/CPANPLUS (or some similar system of automatically dealing with dependencies, e.g. Debian's debs via apt-get), then life will get much easier for CPAN authors...

Aren't you already there? Who isn't using package systems that automatically know how to resolve things? If this isn't the case, then perhaps it's not easy enough yet (I'd argue this is part of the problem). Look at yum/rpm, or apt-get. These are ultra simple for the user.

>> At present there's a direct trade-off between modularity and ease-of-adoption, because if your module depends on a dozen other modules, a large percentage of the perl users out there will give up on the download & install grind before finishing.

While there is some truth to this, I believe the current implementation to be part of the problem. cpanp is better than cpan, but it needs to go further. I'd contrast how easy it is to install via something like Yum, to CPAN. One obvious issue is that most people (even developers) don't want to see the 10 tons of crap scroll on their screen for a CPAN install (although it seems to have gone down to 5 tons of crap for cpanp). If I'm installing something, all I need is some dots .... for progress, so I know something is happening. Then, if there is a failure, tell me. Note, you have to give up on the current cpan paradigm, which was very developer-centric, and do the following: 1. Automatically follow dependencies by default, and 2. Don't show me a bunch of crap. You could make people happy with automatic follow, by first presenting the screen of what you will install with a y/n (just like yum does).

Also, please don't tell me I can configure cpanp to work this way (as I know you can). That's the issue. A newbie won't know how to do this, and will probably break one of the 100 pieces of config. Make it "simple" by default (simple = reduced info, auto follow). Let the developers turn on enhanced detail.

I really urge the CPAN people involved to look at yum and ask "why can't cpanp be this easy?" The answer is it can (and it's getting there). Then you won't have all these arguments like "people are afraid of module systems". They aren't, really, it's not that big a deal. If I have to sit and wait for a download, why do I care if it's downloading modules or one big thing? Most of the computing devices I own do this, and I'm not conscious of it (Mac, PS3, iPhone). Really, it's a solved problem. People want it to be simple. All I care about is having to hit OK a million times, and look at a bunch of technical crap that scares the bejesus out of me. Even if things fail, people still don't want to see that technical mumbo-jumbo. People are afraid of CPAN, because it was made for developers (note I'm a developer, but putting on my consumer hat here). I'd make cpanp behave more like yum by default, but have a "developer mode" that switches all the detail back on. Put it in "dummy mode" and have developers configure it differently. Also, aren't developers tired of seeing all that junk on the screen? I am. It's visually exhausting and mostly irrelevant. Just tell me everything is ok, and a few lines about what you did or are going to do. Put the detail on a log (as cpanp does currently).

Is there a light at the end of this tunnel which is not an oncoming train?
Yes, post what you tried, so others can try to replicate your problems.

OK, your certainty was enough to make me go back and reduce things to figure this out. Since I'm new to cpanp, that was helpful. Because it seemed like the cleanest and best approach, I focused on cpanp "custom sources". Here's what I think I can say with some certainty (test follows):

1. Custom sources determines "newer" version based on lexicographical compare, not version objects.
2. cpanp _will_ find the latest install, provided your version numbers follow lexicographical ordering scheme (which I'm assuming is the older perl scheme). Thus it _does not_ need author/version as I'd said (I think the MiniCPAN method _does_ need this, but did not go back and test). Note that the newer perl versioning scheme (with the "v" on the front) doesn't work this way (example code below). So my guess is that cpanp does not use "version" objects. Thus for cpanp v0.10 < v0.9, which is _not_ true for version objects.
3. cpanp _does not find_ any custom source preqs, regardless of how you specify the version in Build.PL. My guess is it is not even looking for these (although I haven't looked at the cpanp code yet).
4. Custom sources requires one to do "/cs --list" prior to any command using an index, or it doesn't find the index (you get a URI index error). This is clearly a bug. My guess is that "/cs --list" may be populating an internal list. This _may_ be the reason I thought it was necessary to do some of the previous weirdness (write file then update). All one needs to do is, "/cs --list", "/cs --update 1" to update the custom sources (but note "/cs --list" _is_ necessary).

Here's a demo of perl version objects:

$ perl -e 'use version; $v1 = new version("v0.10"); $v2 = new version("v0.9"); print $v1 > $v2, "\n"'
1

Note it printed 1, so v0.10 > v0.9, which is not lexicographical, but the version object order (so we don't have to pad to v0.09 for example).

Next here's a demo showing that while it does install the latest module (provided you follow version lexicographical order), it will not find your prerequisite in custom sources. Note that I tried about 10 different slight permutation on the "requires" string in Build.PL, as described int the docs, from ultra simple, to prepending "v", etc. I couldn't get anything to work (i.e. cpanp never found any of these types). So I'm guessing that it doesn't even look (but I'm new to cpanp so take it with a grain of salt).

# Make empty custom sources dir

    $ mkdir customsrc

# Make dummy module

    $ module-starter --module=My::Module::Test --author="Jane Smith" --email=jane.smith@example.com --builder=Module::Install
    $ module-starter --module=My::Other::Module::Test --author="Jane Smith" --email=jane.smith@example.com --builder=Module::Install
    $ cd My-Module-Test

# Add version to generated lib/My/Module/Test.pm =>  "use version; our $VERSION = qv ('0.1');"
# Add "requires line" to Build.PL => "requires { 'My-Other-Module' => 'v0.1' },"
# Note, I tried many permutations of this, as well as
#   changing the version spec in My-Other-Module.
#   None of these 10 or so tries worked.

    $ perl Build.PL
    $ sudo Build dist
    $ cp My-Module-Test-v0.1.tar.gz ../customsrc
    $ sudo cpanp

    \CPANPLUS::Shell::Default -- CPAN exploration and module installation (v0.84)
    ...

    CPAN_Terminal> /cs --add file:////<path-to-your-local-dir>/customsrc

    Added remote source 'file:///Applications/MAMP/svn/qws-perl-libs/customsrc'

    Remote source contains:

       My-Module-Test-v0.1.tar.gz

    CPAN_Terminal> install My-Module-Test

# installs OK, but doesn't find My-Other-Module
    ...Warning: prerequisite My-Other-Module v0.1 not found.

I feel like the custom sources is very close to what I need, and seems a bit less flaky then I previously thought (well, now I am closer to knowing what the real bugs/problems are).

I still need to figure out how to make it look for the custom source prerequisites.

Just a follow on message to let people know this is resolved, and custom sources works fine as long as you do the following two things:

1. Use "old style" versions (i.e. $VERSION = "1.50"), which have a "2-digit zero padded minor form". Do not use extended versions (i.e. use version; our $VERSION = qv ("1.5")) as this doesn't seem to work.
2. Make sure to refresh both the custom sources and update indexes any time there is a change. The non-obvious method (using a side effect of --write) to do this is:
   1. /cs --list
   2. /cs --write /dir
   3. /cs --update 1
   4. x

You can validate that cpanp can "see" your sources by doing "m Your::Package". If this search doesn't find it, then cpanp won't install it.

Custom sources are much easier than the MiniCPAN approach, in my opinion, if you just want to add your own libs.

OK, I figured out everything I was doing incorrectly, along with some idiosyncracies of cpanp custom sources. I think the newness of cpanp, and some things I was doing in a slightly unorthodoxed manner contributed to my confusion.

That said, there are slight incorrectnesses in the handling of versions, which are probably due to my using version objects. When I look at most version numbers for cpan things they follow a N.NN format (zero padded minor). If most people do this, then things will tend to work (I didn't). Another thing that is slightly incorrect is the way things get updated for custom sources in cpanp. Doing "/cs --write file" has the side effect of making "/cs --update xx" work properly. If you don't do this, then it won't work. Also, if you don't do "/cs --list" then indexes don't get created (and you can't use indexes rather than URIs in your /cs commands). That said, most people probably go through the steps in order as they are done in the documentation and so they never see these bugs.

First off, cpanp custom sources is definitely the easiest thing to use if all you want is to maintain your own mini repository of sources. I originally wanted to go the "Mini CPAN route", but have rethought this. There are likely benefits to this, but "cpanp custom sources" is far simpler, and everything seems to work.

There are three basic problems:

1. Updating cpanp's various cached indexes for the custom sources when something changes. If you are changing modules constantly, you'll have to do these same steps each time. Further, this is a non-obvious, multi-step procedure.
2. Build.PL's version should be of the "x.xx" variety, and not the "perl version object" variety. The perl version object supports a wider variety of version styles, whereas the, e.g. "1.05" style is more of a lexicographical version. cpanp makes assumptions as to the version. For example, a version 0.4 will get coerced into version 0.40. If you specify "0.4" in your build this won't match "0.40". The easiest thing to do is to always zero pad the minor number, everywhere. So your version string should look like "0.40" and not "qv (0.40)" which is the newer style.
3. Be careful where you use "::" and "-" as separators. If something requires a "module" (module search, requires), then you want "::".

By paying attention to these details, I was able to make modules and have them work just like all the other modules (e.g. it would find prerequisites correctly).

Updating cpanp's internal caches was non-obvious. Here are the steps required to do this:

1. /cs --add file:///your-dir # only required first time
2. /cs --list # required to set index 1 to your URI
3. /cs --write /your-dir # required to write your packages.txt file for later step, but also required to get new changes to dir. Note this is not a URI but a file, so we are kind of cheating and writing it to the place the URI refers to (another gotcha). If we don't have direct access we'll have to copy the resulting file to the head directory.
4. /cs --update 1 # If you do not do the previous --write step you can see that --update doesn't work properly
5. /cs --contents 1 # verification, make sure your package is there.
6. x # required to update the indexes
7. m Your::Package # verification 2, make sure your pacakge shows up (if you omit 'x' step above you won't see your package.

The cool thing about cpanp is I should be able to script the update step above so that it's done prior to running cpanp interactively.