• You shouldn't modify modules from CPAN. Otherwise, upgrading removes your changes. If you need to make changes, you should write a wrapper around it. If you can't, you should be working with the author to incorporate your changes back into the mainline.
• If something needs compiling on a machine that doesn't have the tools, then you have a problem no matter what you do. Something on one machine will (almost) never work on another machine, particularly if it's a different OS (like compiling on Linux to work on Windows).
• Subversion is the SCM (source control manager) that I use and that much of the Perl community is moving to. You might want to check out SVK - it's a wrapper around SVN that adds a lot of useful functionality.
• Deployment is always a hard topic. The biggest key is being able to deploy with one command. At $work, we tend to deploy using SVN. But, that's not necessarily the best solution in all (or even some) cases.
• A good testing infrastructure isn't something you can just throw together. Catalyst has an excellent testing infrastructure you might want to look at. It allows you to simulate various requests and see how your application code responds. This is a "Good Thing"™.

I wanted to affirm Dragonchild's comments. Whenever a project gets beyond a certain minimum size (IMO: if it takes more than a few days for a single person to recreate it from scratch), source control is invaluable for managing development, debugging, and maintenance. Adding in some kind of regression testing scheme for every change is also quite handy. Well-constructed regressoin tests are great for finding unexpected interactions, isolating bugs, and so on, even for an environment as robust as Perl's. The only thing I'd add is that a testing and revision control system takes discipline to use well; the value you get from it depends largely on the effort you put into it.

You shouldn't modify modules from CPAN. Otherwise, upgrading removes your changes. If you need to make changes, you should write a wrapper around it. If you can't, you should be working with the author to incorporate your changes back into the mainline.

Not so anymore :D

I don't know for others, but with Subversion it's possible to mix it. You first import some module, then make a brach and change it the way you like. After some time author uploads a new version to CPAN (with hopefuly a patch that you sent to him/her - so there is no need for anything else). You import that as well - and SVN can give you a new version from the author, with your custom code ... How cool is that!?!?

I must admit I haven't tried it yet - but you can find info on that in the SVN book... I think it's under -Vendor Branches-.

If something needs compiling on a machine that doesn't have the tools, then you have a problem no matter what you do. Something on one machine will (almost) never work on another machine, particularly if it's a different OS (like compiling on Linux to work on Windows).

Of course that something compiled for Linux wont work on Windows and the other way around :D I was thinking along the lines of having an older machine with OS that client has (FreeBSD or Linux in 99.99% cases) to compile the code myself ...

---

Have you tried freelancing? Check out Scriptlance - I work there.

You import that as well - and SVN can give you a new version from the author, with your custom code ...

You have obviously not used SVN's merging tools. While very nice when they work, they often don't work, requiring a lot of hand-merging. Frankly, it's much better just to use the CPAN shell to install the version you want to install. Most of the time, installing the latest and greatest will be good enough.

I was thinking along the lines of having an older machine with OS that client has (FreeBSD or Linux in 99.99% cases) to compile the code myself ...

Assuming you're running the same Perl version. 5.6 and 5.8 are NOT XS-compatible. This means that something compiled against 5.6 will NOT run with 5.8 and vice-versa. 5.10 is rumored to be XS-incompatible with 5.8. The solution you're looking for is a list of prerequisites that you (or the client) install on the client machine. Period. Anything else is asking for more trouble than you want to deal with.

---

My criteria for good software:

1. Does it work?
2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

I really don't think modifying CPAN code is good practise, unless it's to fix a serious bug that you then give back to the author.

If I really want to modify the behaviour of a CPAN module and it's not reasonable to subclass (or I couldn't be bothered ;) I usually inject my own subs/methods into the namespace from my own code. That way you don't need to carry around modified CPAN modules. And with any luck, minor upgrades to the cpan module won't break your addins (which are hopefully based on documented API anyway).

My primary vocation is that of Network / Systems Administrator and as such I have to deploy and maintain the applications from developers outside our company. When I read that you modify CPAN modules for deployment to your customers... I couldn't believe it!

The absolutely worst thing I can think of is a developer requiring a hacked development environment on my systems. I don't have any problem with a developer requiring particular versions of modules or interpreters / compilers on the system, but when they modify other authors work that will break other developers code that may use that same module that is in my opinion extremely bad form and as such I would put in a recommendation not to use that developer again.

I know this seems harsh, but when supporting an application environment we don't want to have unidentifiable incompatibility with another program using that same module - and of course if we reinstall that module - then the first program mysteriously stops working.

Standards MUST be complied with, if you need to change the operation of another authors module, change it in YOUR code, or if appropriate contact that author and work with him, and if it is an orphaned project, see about taking over the code and deploy your revision of the code to CPAN, properly documented of course.

I understand and I really agree.

I simply put modules that are not "standard" in the "local" libs/ folder that I add to @INC (just in application - not for whole server) - so it's only used by my application.

---

Have you tried freelancing? Check out Scriptlance - I work there.

But the best benefits (IMO) are the longer-term benefits. You'll be working on dozens of projects, and when you come back to one after not touching it for a year, you'll be amazed at what you don't remember about it. (In fact, I'm occasionally surprised when I'm asked to make changes to something--and I find that *I* wrote it years ago. I totally forgot everything about it.)

In cases like this, you'll find that it's *much* easier to change the system. You go in, read the code, make the changes, and the testing system will remind/alert you to assumptions you made this time that are different than last time. So the better your test coverage, the easier it is to make a patch "that just works".

Another thing I like: You know how sometimes a line of code or a module looks like it could be simplified or cleaned up? So you do it, and you find that it fails in some way. So you put it back the way it was. Then another person comes in and makes the same observation. (S)he comes in, does the same cleanup/simplification, and has to fix it.

Putting in a test makes it easy for you to prevent this and other types of bug regressions. Before you make a patch, make a test that forces the bug to occur, then fix it. Then if someone changes the code to re-introduce the bug, they'll be instantly alerted to it.

My final reason for enjoying test suites: Sometimes you want to make a change to a subsystem. You make it and the test suite shows you dozens of breakages. This allows you to find dependencies that you can correct, so you can further simplify your code. Some of those breakages will be unavoidable, so it will help document some requirements for your subsystem. Also, for fun, you can try to predict the breakage. This can help you keep an eye out for unnecessary interdependencies between subsystems.

There are many other reasons to make testing a primary concern, but I can't think of 'em at the moment. But you'll find them soon enough once you integrate testing into your development regime!

--roboticus

Usually I create an installation package of perl and the modules I use:

• For pure windows it's easy with the activestate zip version and ppm/ppd
• For Linux it's not much more difficult because there is nearly always a gcc installed; just some external libs sometimes cause problems
• just for solaris, this easy way often doesn't work because the compiler suite usually isn't available).

But I didn't really solve the problem with OS independend installations for the whole perl version with modules.

If I need to upgrade a module, a testsuite for my code helps me finding errors. I can't trust module authors not to change interfaces any more (recently happened to me in Readonly v1.01: Readonly \$scalar => 20 to v1.03: Readonly $scalar => 20, or Mail::Mailer, where the option 'test' was replaced with 'testfile')

The only time I had to change a CPAN module (needed to change the Makefile of DBD::SQLite from UTF8 to ANSI because of an old Tk version (8.04?) which couldn't handle UTF8 correctly) I additionally saved the whole perl installation within a version control system. Since the only target was windows, it worked well.

Finding the right deployment strategy takes some practice. I'll tell you some basics of how we do it at http://plusthree.com/, for a fairly large Perl application.

First, we keep the code in Subversion. In addition to some obvious benefits of avoiding overwriting each other's work and keeping long-term undo, it's the only way to deal with needing to keep a released version working with occasional patches while you build a new version. This is done with branches.

Next, you need to make a release somehow. We have a script that build release packages. They aren't much more than a .tar.gz file, but some things that are only relevant for development are left out, and care is taken to avoid over-writing config files.

We move that to our server, where we unpack it and run an automated build script that builds all the included CPAN modules. Then we run upgrade scripts which make any required database changes between releases.

We include the CPAN modules in their normal compressed state and automate the whole unpack, Makefile.PL/Build.PL, install business. We install them into a local directory, not the site_lib on the machine, and point our apps at that directory. This means that we always know exactly which version of a module we're using and never get burned by a change in a module's API on an upgrade.

There have been cases where we needed to modify a CPAN module. In that situation, we move it to the source tree, at least until a later version that no longer needs our changes is released. (And yes, we share our changes with the authors in these cases.)

Since we're dealing with dedicated hardware, not a cheap shared host, we control the environment as much as possible. We build our own perl, so we can count on knowing the version and the compile options and not be surprised by a special Red Hat patch or a missing Scalar::Util feature. We also install a specific version of the database, so we know what to expect from it too.

If you're building something to run on shared hosts, or your project is small, then this may be overkill for you. It's been really helpful for us though. You can see most of this in action in the Krang CMS project, which shares many of the same build and deploy approaches.

Yes cheap shared hosting is what makes problems to me most of the time. As I'm still a small fish and don't want to cash ~ $100+/month for dedicated server I took one of those VPS for my needs. I also sugest that most of the customers should take at least VPS (they start at $20/month) if for nothing else than mod_perl/FastCGI posibility.

The thing is that applications being developed are also to be sold to 3rd parties of which most will have shared hosting. And under those conditions only pure Perl modules will work ...

Anyway - in case you want to start using a new module, you first import it in the repository, then do a new export/chekout (script that builds releases handles it). And then you can use it, right?

Are there any articles/books/whatever on this subject? But something that can be applied to Perl?

---

Have you tried freelancing? Check out Scriptlance - I work there.

I'd expect even a shared host to have a compiler installed.

When adding a new module to the repository, we just check the downloaded CPAN package into a directory and then run the build script. It finds all CPAN packages in that directory and builds them.

Most of the articles about software deployment and configuration management tend to be about Java, since many LAMP developers do this kind of thing in sloppy ways without much thought. It's only when you get into larger apps that you realize the necessity of thinking through infrastructure like this. However, you may find some useful things in the archives on the http://onlamp.com/ site.

I have been looking for ideas about how to automate Perl app deployment and module managment.

currently, we have all the commonly used modules (such as CGI::Application and other in-house modules) in a central perl lib directory and all apps use it from there. problem may arise if a module gets upgraded with API changes and such. but i think testing may help us stay away from trouble.

to your reply, I am wondering why not unpack all needed modules locally once for your app in development and save to cvs. when deploy to the production, just cvs out. be done with it? (the only reason i can think of not doing it is unless you need to deploy to different OS). rebuilding same modules on dev,QA,production sound tedious.

or, you can keep all unpacked modules in a central location, copy to new app when needed without rebuilding it for every release.

thoughts?

Compiling once the way you describe will only work if you are sure you never need to support another OS, different hardware, or a different version of Perl. These things will also change over time. You will want to upgrade your OS and suddenly find you have to recompile all of those modules by hand. Building an automated installer makes this easy, and has made it easy to deal with adding new modules.

I use Module::Build, Module::Starter, CPAN::Mini and CPAN::Mini::Inject to keep a local CPAN mirror. With CPAN::Mini::Inject, I can add modules that I wouldn't want to post to the real CPAN.

With Module::Build and Module::Starter, you can generate complete CPAN-ready modules with almost zero extra hassle. Once I learned about them, I kicked myself for not doing it earlier. There is soooo much to gain for so little extra work. Documentation and testing become totally easy.

CPAN::Mini::Inject doesn't support bundles or an authorid that isn't registered with CPAN. I was working with the author on adding those features, but he hasn't responded since I sent him my code... I should bug him again...

In a semi-unrelated note:

Last time I looked into Subversion, it was Linux only. Has that changed? I still use CVS because I work in Linux, OpenBSD and Windows. If Subversion will work on those platforms, I'd love to hear about it.

Guess I'll have to check out those modules you mentioned ... Thanks for the tip.

Yes, Subversion works on Windows too. Not sure about OpenBSD ... Anyway there is an companion project called TortoiseSVN (same as TortoiseCSV but for Subversion) which gives you Windows Explorer integration. So you have right-click context menues with SVN commands ...

---

Have you tried freelancing? Check out Scriptlance - I work there. For more info about Scriptlance and freelancing in general check out my home node.