http://qs1969.pair.com?node_id=719239

RAS230 has asked for the wisdom of the Perl Monks concerning the following question:

I've been messing around with perl on and off for a few years now, but never for anything too serious, mostly automating basic sys admin type tasks. I thought modules were mostly for code reuse, and anytime I found myself wanting to add a subroutine to a script that I'd already written for an older script, I'd consider throwing it in a *.pm file.

Now I've got a small handful of *.pm files that each hold several subroutines related to a particular type of task and I use those all the time in my scripts.

If I want to share a handy script with someone else, all I have to do is send them the script and the .pm file(s) it needs or I could take a few seconds to copy the subroutines I need out of the .pm file, paste it into the script itself and modify a line or two so I that can send the whole thing in one file. I thought it was all pretty handy, until I saw CPAN. Now I think I'm doing it wrong.

Sure I'd heard about CPAN, and how it was this great place where people shared all their most useful perl code, but I wasn't really interested in it. I use perl scripts everyday but I don't write new ones all that often. I wanted more experience with the language before I started to mess around with other peoples code. A little bit of "re-inventing the wheel" is great when your learning right?

Recently I've been starting to use perl more and more often, and I've been poking around here, and thought I would finally take a look at some of the modules at CPAN as well to see what I could learn from them. Many of them do look interesting but none of the ones I saw look likely to help me learn perl. It looks like I'd learn more at CPAN as a C programmer. Even the perl scripts on CPAN require modules that were not written in perl.

Why am I compiling perl modules when I can't even compile the scripts that use them? Are all these modules written in other languages because they really cannot be done in perl alone? Where (besides here) is the site where people share useful perl code?

when I consider my own simple (although clearly misguided) setup for reusing and distributing perl, I have to wonder why real perl modules are considered to be such a good idea. I've never had to worry about ppm, dependencies, repositories, or make/compile errors before.

One of the things I love about perl is that I can replace my shell scripts and batch files and use one perl script for both systems. If I wrote a script using one of these modules how could I send it to my friend using windows? I'd have to include a huge set of directions (and probably include a link for nmake) and then hope it (and all it's dependencies) will be supported by the version of activeperl they run. In the past it was as easy as saying "unzip the .pl and .pm file into the same directory".

I really thought *the* website for perl code would be much less complicated.

If I've just got the whole concept of perl modules wrong, what should I be doing with my collection of subroutines?

I was pretty excited about using modules to start getting more out of perl, now I just feel disillusioned and even more unsure of myself.

  • Comment on Have I misunderstood the point of modules or just CPAN?

Replies are listed 'Best First'.
Re: Have I misunderstood the point of modules or just CPAN?
by Fletch (Bishop) on Oct 24, 2008 at 05:12 UTC

    Keeping things "pure Perl" is a valid design constraint, but there's stuff that the vanilla base perl can't do that require external resources. Interfacing with native C libraries, for one instance, which already solve a particular problem pretty thoroughly (e.g. ImageMagick). Another reason why things aren't done in pure Perl is that often there's a speed advantage to manipulating things directly in C.

    They're a good idea because they work, they're tested, and they get the job done quickly. And with PAR you can distribute them fairly easily without worrying about the other side's configuration or dependencies or what not.

    So no you're not "wrong" to want to stick to pure Perl, you've just chosen to pay more attention to different constraints. Just be aware that there's other approaches and ways to avoid the downfalls you've noted.

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

Re: Have I misunderstood the point of modules or just CPAN?
by b10m (Vicar) on Oct 24, 2008 at 09:28 UTC
    "Are all these modules written in other languages because they really cannot be done in perl alone?"

    The good thing about Perl is TIMTOWTDI, and the same goes for CPAN. If you want a pure perl way to handle CSV files, go for Text::CSV_PP, if you don't care about it being pure perl and you'd fancy speed (like me), you'd go for Text::CSV_XS.

    I don't see the problem here. Perl isn't great at everything. I'm not writing up a Perl program when find + xargs can do it better/easier. I'm not building something myself something when bzgrep will be faster/easier. (Although I do use ack more often than grep nowadays).

    Perl isn't always the right choice. Often, yes, but sometimes other languages come in handy too. In the above CSV example, the XS part will most likely speed it up. I'm really curious how you'd think something like Net::SSLeay or ImageMagick would be created without compiling evil non-Perl code, though. And what modules using non-Perl code modules trouble you this much if I may ask?

    --
    b10m
Re: Have I misunderstood the point of modules or just CPAN?
by perrin (Chancellor) on Oct 24, 2008 at 15:53 UTC

    Hi,

    Glad to hear you're playing with modules and checking out CPAN.

    Most of CPAN is not written in C, but some of the most useful modules on it do have some C code. A good example would be DBI. Usually when modules use C it's either because the job requires the performance of C or because a handy C library already existed and the author wanted to make it available to perl coders. But the vast majority of CPAN has no C code. If that doesn't match your experience, maybe you'd like to tell us what modules you've been looking at?

    Your general question seems to be why the modules on CPAN are more complex. I'm guessing that your modules may not have separate package names and just import all of their subs into the main program when used. They don't have Makefile.PL installers like CPAN modules do. And they don't track dependencies between themselves.

    The answer has to do with scale. For a very small amount of code like you have, the complexity can be managed without help. When you start to have large projects with thousands of lines of code, you need the things CPAN-style modules offer to keep it all straight. The standard installation and dependency tracking makes it possible for the whole perl community to use and contribute to CPAN.

    If you're wondering which practices you should try to emulate in your own code, you might start with package names. Small modules that you only use locally don't usually need to have installers or dependency tracking.

    BTW, there are tools for building an executable with all dependencies if you want to send a complex thing to friend on Windows without making them install modules from CPAN.

    I recommend you check out this book, Writiing Perl Modules for CPAN, by samtregar. It's a free download and will answer your questions more fully.

Re: Have I misunderstood the point of modules or just CPAN?
by Anonymous Monk on Oct 24, 2008 at 07:03 UTC
    Programming is a lot like building cars ... You've living in the years before WW2, you're a farmer, and you've built yourself a nice jalopy, it gets you where you need to go, you can fix it when it breaks, all you need is a hammer and some nails. You loan it to your friends and neighbours, and no one has a better car in town. Now you've moved to the city, not just any city, the biggest city in the world, the city of the future, the giant-super-hyper-megalopolis , CPAN, where everyone's got hybrid-powered flying/swimming/dancing cars that chit-chat while they massage your behind, all while making you coffee, cooling you off, and avoiding pot holes and reckless drivers. Your jalopy doesn't even have a roof, seatbelts or shock absorbers, you've never even heard of fuel injection and you only get 3 miles to the gallon. You're scared and confused, don't understand why all the CPAN cars have all these things. You're not gonna learn all the whys and hows by looking at the cars, you gotta go to school boy and hit the books :)

    Putting cars together is a lot harder than just driving them, and if you're building a car for Joe the milkman, don't try to turn him into a mechanic, give him a single file he can just double-click and follow directions, nothing else to copy, nothing else to install.
    :)

Re: Have I misunderstood the point of modules or just CPAN?
by Your Mother (Archbishop) on Oct 24, 2008 at 16:39 UTC

    I think most CPAN hosted modules have no C/XS involved at all. What you will see and might make it look otherwise is that many of the best, most popular modules do have some. Their popularity/utility makes them prime candidates for optimization. Things like XML::LibXML are interfaces to really great packages not written in Perl. The success of these things is predicated on most everyone being in development environments where compiling the stuff is not difficult.

    Perl is the only reason I'm a hacker so I mean no slight at all when I say that without the CPAN Perl would come off somewhere between PHP with Asperger's, Anarchist Java, and the abandoned feral child of C+Unix.

    As to the CPAN itself, there are mountains of detritus in there but there is also some of the best software engineering around; ranging from brilliant generalization like Moose to completely specific problems like Spreadsheet::ParseExcel. There are hundreds, if not thousands, of nodes on PM discussing the nature of the CPAN.

    MST gave a lightning talk at a recent conference about why the detritus is good for the environment too (audio is not good but I found it worth the trouble): You're a terrible person and should never contribute to open source software, ever.

Re: Have I misunderstood the point of modules or just CPAN?
by planetscape (Chancellor) on Oct 24, 2008 at 18:01 UTC
    when I consider my own simple (although clearly misguided) setup for reusing and distributing perl, I have to wonder why real perl modules are considered to be such a good idea. I've never had to worry about ppm, dependencies, repositories, or make/compile errors before.

    If the way you are doing things now Works For You™, then why do you think there is anything wrong with the way you are doing things now?

    Membership in (and adherence to the standards of - if any) CPAN is not mandatory. No one will make fun of you for doing what works. And if they do, they are losers who should be fed to a dragon. ;-)

    HTH,

    planetscape
Re: Have I misunderstood the point of modules or just CPAN?
by RAS230 (Acolyte) on Oct 28, 2008 at 04:14 UTC
    That 'writing perl modules for CPAN' book was a huge help! It cleared up a lot of questions I've had in the back of my mind.

    As I spend more time looking around CPAN I'm finding more and more modules that are written with only perl. It seems like it was just my luck that the ones which initially caught my attention seemed to use a lot of C. I'm even finding pure perl alternitives for some of them or modules like Geo::IP which seem to include both.

    It may be worth mentioning that I have nothing against non-Perl code in general. I even messed around a bit in C (12+ years ago). I was just not expecting to find it on CPAN. I had hoped that I could see how other folks were using perl to do the same sort of things I've been doing with it. When I saw all the non-perl going on I was starting to wonder if that would be possible. No more worries there!

    A lot of people seem to run into trouble installing modules, and so I also had concerns about how easy/difficult it would be for others to install a script if it needed modules from CPAN. I'm less worried about that now that I know about PAR and have a better understand of how CPAN modules work in general.

    I'm guessing that your modules may not have separate package names and just import all of their subs into the main program when used. They don't have Makefile.PL installers like CPAN modules do. And they don't track dependencies between themselves.

    That is exactly the case. Like I said, before now I haven't really paid much attention to the whole module side of perl. The fact that my modules are so different from anything I saw on CPAN was starting to make me think I'd missed something important.

    In some cases just doing what works for me would be enough, but I've been able to get something working only to find out later that it was put together in some abhorrently wrong manner that wouldn't work for anyone else, shouldn't even be working for me or doesn't work anywhere near as well as it would if I'd only done it correctly.

    I may not ever write something that's used by more than 2 or 3 people, but anytime I do write something I try to imagine how it might look to someone else if they had to go in to fix/update/build on whatever I've done. Now that I want to start doing more with perl I figured it was worth checking with folks who know better.

    Now that I feel much more enlightened I think I'll be safe if I stick to what I've been doing. Hell, after checking out the book and that video, I can see myself adding a module (or at least a script or two) to CPAN at some point in the future.

    Thank you all!