Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Top 11 (GOOD) reasons not to use someone else's Modules

by miketosh (Acolyte)
on Apr 23, 2012 at 14:53 UTC ( [id://966607]=perlmeditation: print w/replies, xml ) Need Help??

I have often seen (sometimes rude) responses to people asking questions that say, "Just install module XYZ". This may not always be the best option, and I cringe when I see that. Here are my reasons for resorting to other peoples' modules last.

  1. LEARNING I won't learn anything new if I just reuse code. Like the old Proverb, "Give a man a fish, and feed him for the day. Teach a man to fish, and you feed him for life.".
  2. PERFECTION I'm a perfectionist, and fear someone else's code may be sloppy or not as fast as it could be. Since I often have most of my code written already, I don't want to spend time combing through something else.
  3. PURPOSE The code does something similar to what I want, but not exactly. I'd spend just as much time to parse the output as I would to rewrite the bits I need.
  4. APPEARANCE I don't own any of the boxes I'm trying to do something on. "Installing" a module is a bigger perceived change than to just craft a simple script to use. For some of those boxes, the system owner who gave me temporary access may not like me adding a ton of stuff and may be less willing to share a shell next time. Since this is more often than not my customer, I do what I can to work within their preferences.
  5. PORTABILITY I like to keep everything in one file so that I can move it to another box that may or may not have the same modules, and who knows what versions. If I share my script with someone, I may forget its dependencies when moving it.
  6. CPAN ACCESS (And this is a big one) Every system that I use is behind numerous firewalls that all restrict access to everything. To install a Perl Module, I'd have to copy all text from each file, and paste it on the target system via an xterm in a similar file structure. Cumbersome.
  7. COMPILING Most systems I use don't have compilers. Cannot use Perl Modules that need to be built, which for me has been about half of them that I've tried.
  8. COMPLEXITY It is counter-intuitive to me to install a Perl module for a one-time script. I don't often write Perl code, as much as I run one-off perl commands. Installing a Perl module in a subdirectory of $HOME, then remembering to "use lib qw(/home/me/lib/)" every time I run a command is a lot more work that it is worth.
  9. SIZE Requiring lots of Perl modules can eat up memory. You may say memory is cheap, but these are government boxes, again not in my control. We have to run MySQL and Apache for 200+ concurrent users at all times in 2GB of memory. Each PM eats up memory and takes time to load.
  10. LICENSING If the module is GPL, and I'm delivering this as a finished system under a restricted license, I have to get a review from Legal. Ugh.
  11. JOB I'm paid to write code, not download modules. If I'm asked to write something but get stuck, it would be cheating if I just downloaded something else. I'm not looking for the answer, I'm looking for a hint.

Off the soapbox and back to work.

Replies are listed 'Best First'.
Re: Top 11 reasons *NOT* to use someone else's Perl Module
by jhourcle (Prior) on Apr 23, 2012 at 16:05 UTC

    It's nice that you're willing to share, but a few of them seem counter productive to me.

    1. Most of us have jobs that are about getting things done, not specifically about learning. If we have free time once things are done, we can take the time to learn, so code re-use means stuff can get done more quickly, and we can use code that's already been used/tested by at least one other person, reducing the time needed for maintenance ... which may free up time for learning.
    2. This may be contradicted by your 'learning' and 'complexity' comments. After all, if you don't write Perl that often, are you going to be as familiar with all of the possible optimizations as someone who does?
    3. It depends on the complexity of what it is that you're trying to do. And sometimes it's easier to subclass something to override the output than try to parse it after the fact.
    4. I can see part of that argument ... the bigger problem I would think would be those cases where you don't have shell access. (every tried debugging via e-mail? )
    5. There was a talk at the DC-Baltimore Perl Workshop last weekend on 'Hooking @INC' that covered how get details of what's required to run something and what still needs to be installed on a given system.
    6. If you can't SFTP or SCP a tarball in, you may be able to send it to the sysadmin to place on the machine for it. If you're copy & pasting in each file, something is horribly wrong.
    7. A valid argument -- I've worked plenty of places where the production systems had no compilers ... but in every case, there was a development server that did, and you were expected to build there, test, and then push the updates to the production server.
    8. If you never do it, you'll never get in the habit of doing it ... once you do it a bit, it's second nature. (or, you just set up templates so you don't have to remember)
    9. Might be a valid argument, as I don't know your exact system. But memory *is* cheaper than programming time for anything but trivial work when you start counting overhead (assuming the machine has a slot free).
    10. You have to get a review *every* time? Something's wrong with your contract if they can't give you clear guidelines on what you are or aren't allowed to do. (eg, are you selling them the code, which may be a form of distribution, or are you creating the code for them as an employee or contractor?) The only time it's ever been a problem for me (as a contractor) was when the ExtJS folks changed from BSD to LGPL to GPL3, and then gave their interpretation of GPL in which they claimed the backend CGIs that interacted with their code was bound by GPL ... but you make an interesting point. Anyone know of any CPANTS reports that show which modules are GPL3 only and not also releases under a more permissive license?
    11. But yet you're paid to learn on the job? I'm guessing that you're like most of us -- you're given the end task and maybe some restrictions in how to implement it You may set some other limitations for whatever reason, but 'being paid to write code' is a flawed argument. If we took it far enough, we could also say that you're cheating by using a high level language like Perl, and not having written your own language.

    (and I'm not being paid to post on here, so I'm going to have to reflect that on my timesheet ... blah)

      (and I'm not being paid to post on here, so I'm going to have to reflect that on my timesheet ... blah)

      And I'll have to stay 15 minutes later today for reading your response! ;-)

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by moritz (Cardinal) on Apr 24, 2012 at 06:49 UTC

    Some of your reasons sound convincing on the surface, but they neglect one huge factor altogether: scaling.

    If you establish a procedure by which you can distribute CPAN modules to the production machines, you have access to many modules. Establishing the procedure is O(1) (you only have to do it once), but reimplementing the modules that you otherwise can't install from CPAN is (at least) O(n), ie must be done for each module.

    The same scaling argument goes for maintenance costs: you can let them (ie the CPAN authors) do the maintenance, or you can do yourself. Which option do you prefer? Which option does your employer prefer? Is it your job to maintain reimplementations of maintained CPAN modules?

    PERFECTION I'm a perfectionist, and fear someone else's code may be sloppy or not as fast as it could be.

    Being guided by the mere fear is irrational. Just look at the code, the tests, the ticket queue. If the code is actually sloppy, slow and bug-ridden, then you have a good reason not to use that module.

    But beware, there is a common pitfall. When people approach a domain that is new to them, they often look at software that deals with that new domain, and think "oh my god, this is over-engineered, overly general and complicated crap; I could do much better". And then they reimplement everything in a cleaner and much simpler way, until they find out that their solution doesn't quite cover all cases, and that handling them all would require much of what the originally rejected module did.

    The lesson to learn is that a reimplementation of a module often takes much more time and effort than initially thought. The main reason is lack of domain knowledge. The main (BAD) reason for reimplementing modules is overconfidence in your domain knowledge.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by Tanktalus (Canon) on Apr 23, 2012 at 18:12 UTC

    1. I get some pay for education. Not that much.

    2. I want perfect code, too. That's why I rely on others who have spent more time on the problem than I have available to provide one for me.

    3. I've noticed that at least 50% of the time where my goals and existing modules diverge, it's because my design is wrong, not the modules. Once I realised that, I learned a lot more. (See #1.)

    4. I prefer to correct the appearance than to work around it. (See #2. Misjudgement is imperfect.)

    5. Module::Install seems to work for me.

    6. If you can get to the system, you can use scp to the system. Also, I prefer checking the desired modules' tarballs to my version control system, which then gets me through all the firewalls. At least as well as any of my other code.

    7. Our target systems don't have compilers, either. We build on a build machine, and deploy on test machines.

    8. I had this one just last week. Someone decried how hard it was to monitor a process with an async process. But because I already had AnyEvent installed, I whipped together a proof of concept of how easy it was in about 20 minutes and about 44 lines of code (with plenty of whitespace). I had AnyEvent installed to manage parallel ssh calls, but it made this one trivial. Many modules have multiple uses.

    9. Ok, you got me here. My target boxes have 16 CPUs and 30+GB of RAM.

    10. Don't grab GPL-only modules. Most modules on CPAN are dual-licensed: GPL or Artistic v1. Some are only Artistic v2. And I've not encountered any that are GPL-only. And I'm going through the legal review right now.

    11. You're paid to write code? Where do you work? I'm paid to deliver solutions. Often that involves code. Often it doesn't.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by tobyink (Canon) on Apr 23, 2012 at 23:49 UTC

    LEARNING and PERFECTION - surely these two points are mutually contradictory? If you're still in the process of learning something, then your code is likely to be naive and suboptimal. You may have missed several edge cases. If you're aiming for perfection, then you shouldn't be using the code you wrote as a learning exercise in production.

    CPAN ACCESS If you need to upload code in a cumbersome method, surely that applies equally to code you've copied from CPAN, and code you've written yourself? I can't imagine a situation where copying and pasting code could be more cumbersome than typing new code.

    COMPLEXITY It may well be counter-intuitive to you to install a Perl module for a one-time script. But for me, it's counter-intuitive to write reams of code for a script that's only going to be used one time. Better to write my code in modules that I can reuse in multiple scripts; better still if somebody else has already done it for me.

    LICENSING The majority of CPAN modules are licensed under the same terms as Perl itself. If their licensing terms are a problem, then using Perl at all is likely to be problematic.

    JOB I'm paid to solve problems. If I can reuse other people's code and solve a problem faster, then I can solve more problems.

    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
Re: Top 11 (CAPITALIZED) reasons not to use someone else's Modules
by Anonymous Monk on Apr 23, 2012 at 16:18 UTC

    And here are 11 sometimes rude responses.

    1. LEARNING § this entirely valid point is only entirely valid if the only point is learning; i.e., if this an excuse to do code you expect to be production worthy or vendible, this sloth has a girlfriend, your point is invalid.
    2. PERFECTION § You've confused perfectionism with narcissism. Perfectionist seek the best possible answer, not the best answer they are capable of delivering.
    3. PURPOSE § You don't parse output of things you want to change, you write patches.
    4. APPEARANCE § this is an ideal point for the Kardashians, and businesses bent on irrelevancy.
    5. PORTABILITY § the difference between a file and a folder is insignificant; i.e., everything in one folder is just as "portable" as everything in one file and a bazillion times less confusing for any significant amount of well-written code. Badly written code does indeed tend to be easier to navigate in a single file though.
    6. CPAN ACCESS § see PORTABILITY and a hundred threads on PerlMonks regarding this topic.
    7. COMPILING § see PORTABILITY + pre-compile it on the same system/hardware for binary compatibility.
    8. COMPLEXITY § The instructions for getting non-root local::lib and cpanm are trivial and are basically fix and forget.
    9. SIZE § Without benchmarking, this is bias. Without analyzing which modules vs what code, it might also be upside-down.
    10. LICENSING § Legal issues are outside the scope of dev advice, so, you win this round, Greater Bandicoot!
    11. JOB § Cheating? Installing Moose breaks neither the laws of God nor *man. The laws of tye and BrowserUK however...

    Off the soapbox and back to making work less unpleasant for the devs who have to maintain the code I write.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by chromatic (Archbishop) on Apr 23, 2012 at 17:35 UTC
    I won't learn anything new if I just reuse code.

    Better give up Perl then. Start with ISBN 0321486811.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by crashtest (Curate) on Apr 23, 2012 at 19:05 UTC

    Most of the points you list deal with the complexity of integrating and managing 3rd party software (specifically CPAN modules). The rest of it speaks to a preference for just doing things yourself. You're not the first to feel this way about either of those two subjects. I'd submit that your list of arguments is based in your distaste for the former and enjoyment of the latter.

    You make a number of valid points about the pain of dealing with CPAN modules. I've yet to meet someone who enjoys wading through someone else's code to figure out how an undocumented edge case is handled. Nobody likes figuring out a deployment/upgrade strategy for a large set of installed modules. People might get physically ill after participating in a long email thread with their company's legal department (ha). None of this stuff is fun, while writing your own code clearly is. You're not on PerlMonks if you don't derive deep satisfaction from crafting a chunk of software on your own. Everyone loves hacking.

    The problem is, dealing with the distasteful stuff like managing CPAN modules is many times more efficient than writing everything yourself. You are dead, dead wrong on point 11 - you are definitely not paid to "write code" (at least I hope not). Your job is to deliver reliable software efficiently (in time, cost). The unfortunate truth is that one has to deal with the unenjoyable parts of $work to find this efficiency, at the expense of doing the fun things like hacking away at code and learning new things.

    One of Perl's great strengths is the breadth and quality of user-contributed software (on CPAN). Like anything else large, it takes time to master its complexities (some of which you listed), things like managing dependencies, licensing issues, gauging a module's quality, etc. But if you do, you'll be able to deliver a better product, and your employer (me, a US taxpayer!) will thank you.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by JavaFan (Canon) on Apr 25, 2012 at 14:33 UTC
    Note that all of the arguments can be applied to perl itself as well.

    1. LEARNING: Have you learned how to write hash functions, deal with signals, IO, and how to create a regexp engine? If not, it's better to use C and learn all those things than to use Perl.
    2. PERFECTION: Perl is slow. A perfectionist would write their stuff in C and have most of its stuff working orders of a magnitude faster.
    3. PURPOSE: many of the constructs Perl offers me don't exactly do what you need -- you'd be better of writing C primitives.
    4. APPEARANCE: Ever tried installing Perl and the myriad of modules and files that come with it? Just the endless list of questions being asked at Configure time alone. No, in other to not lose your precious access, it's much better to install a compiled C program (one file!) instead of the dozens and dozens of Mb of a Perl install.
    5. PORTABILITY: Perl won't run on boxes that you cannot compile on or cross-compile to. But there'll be some platforms your C program can be compiled on (or cross-compiled to) where Perl doesn't. So, for portability reasons, C program are superiour to Perl programs.
    6. CPAN ACCESS: You don't need any for a C program. Not even to install Perl from.
    7. COMPILING: Well, you got me here. If you don't have a C compiler, you cannot compile a C program. But wait, you cannot compile perl then either! So, this one will be a tie.
    8. COMPLEXITY: It doesn't make sense to install Perl for a one-off script, and then to remember to use /home/me/bin/perl each time you run it.
    9. SIZE: Buhahhaha. Perl is a memory hog, and that's before it's doing anything. People have written entire OSses with a full POSIX tool chain in less memory than Perl even needs to run perl -e "1".
    10. LICENSING: Perl gives you the option to either use the GPL, or the lawyers-field-day Artistic License. Legal may prefer you sticking to your C compiler.
    11. JOB: You're paid to write code, not to download perl. That would be cheating. Better build your own table and chair, given an acorn to start with. Anything else would be cheating.
    I think C beats Perl 10.5 - 0.5 using your "reasons".
Re: Top 11 (GOOD) reasons not to use someone else's Modules
by Anonymous Monk on Apr 23, 2012 at 15:51 UTC

    Off the soapbox and back to work.

    Where do you work, I'd like to avoid that place?

    The Obvious Joker

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by Argel (Prior) on Apr 23, 2012 at 21:51 UTC
    If learning is your goal then mention it up front so you get the desired answers. Beyond that, I could see where complexity comes in if a deadline is looming, but otherwise, I would say that politics and legal concerns are normally the only two valid ones.

    Elda Taluta; Sarks Sark; Ark Arks
    My deviantART gallery

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by sundialsvc4 (Abbot) on Apr 23, 2012 at 21:10 UTC

    I respect your right to offer opinions, but I don’t share any of them.   And if those opinions were in the majority, we would not see this tag-line on search.cpan.org:   (today...)

    78,111 Uploads; 24,694 Distributions; 105,940 Modules, 9,641 Uploaders.

    If nearly ten thousand people (software professionals all ...) have produced and freely contributed more than one hundred thousand modules to CPAN since its inception, I think there must be some very compelling reason for it.   I don’t think that any of those people share your opinion, either.   But I daresay they do share mine.

    Nevertheless, this is a public forum, and thank you for your opinions.

      (software professionals all ...)

      Oh, if only that were true.

      I'd estimate that on average, 4 out of every 5 modules selected at random from cpan has serious shortcomings. Ranging from the crude, unidiomatic replication of built in functionality to plain old badly coded and broken crap.

      Whilst the remaining 20,000+ "good" modules leaves cpan with probably the greatest concentration of freely available, quality open source software there is, the blanket acceptance and promotion of "everything cpan" by people like you, does Perl far more harm than good.

        I'd estimate that on average...

        That's not scientific rigor.

        ... serious shortcomings. Ranging from...

        That bucket lumps together a lot of categories. Some of them are even damning.

        ... the blanket acceptance...

        You're putting words in sundial's post.

        ... does Perl far more harm than good.

        That conclusion is difficult to support.

        ... cpan ‘is’ probably the greatest concentration of freely available, quality open source software there is...

        Seems fair to me to assume that a software professional will use professional judgment to decide what to use and when, and which of your identified shortcomings matter in any given set of circumstances.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by Lady_Aleena (Priest) on Apr 29, 2012 at 06:19 UTC

    Hello miketosh. As a dabbler in Perl, there are only a few things that I can touch on here, and I may repeat what has been said by the others.

    1. Learning: How are you learning Perl by ignoring one of the best aspects of the language? Modularization is probably the best parts of Perl. Reusable code is almost the back bone of Perl. I would be totally fed up with Perl by now if I could not reuse the code I wrote or use code written by others who know more about Perl than I do. Also, not using the modules written by others keeps you from learning too. Learning to use other people's code could be a major headache, but it could teach you a thing or two about Perl that you did not already consider.
    2. Perfection: If you have been writing Perl for any length of time, you have probably come across the concept of "there is more than one way to do it" also known as TIMTOWDI. As long as there are many paths to the same result you will never find perfection in your code. I am constantly tweaking everything I write, even those programs that I wrote with the help of others' modules. Perfection is not entirely possible as long as TIMTOWDI is around.
    3. Purpose: Just because a module does not do exactly what you want does not mean that you can not find a way to bend it to your current task at hand. One of my favorite modules of all of the modules I have found is Lingua::EN::Inflect. It may not do everything that I want it to do, however, I take what it can do, use a little imagination, and manage to get the result I want. The purpose of modules is to make things easier for you as long as you have the imagination to make them work for you.
    4. Appearance: If Perl is already installed where you are working, then guess what, you already have modules at your disposal that install with it. Take a look at the list of core modules. Since they are already there, go ahead and use them.
    5. Portability: I am confident that there is a way to make your code portable without you having to redo the work of someone else. All you have to do is go to the search box above and use it.
    6. CPAN access: I can not speak to this issue, since it has never really been one for me.
    7. Compiling: See CPAN access.
    8. Complexity: Never ever say that there is such a thing as a one time script. I said that once and then a few days later I bit my tongue. No matter how small or large, everything you write has the possibility of creeping back into your life when you least expect it to do so. You might not use the exact same code, but it will be close enough where only a few tweaks can make it into a useful reusable subroutine nestled in a module.
    9. Size: See CPAN access.
    10. Licensing: See CPAN accesss.
    11. Job: You are probably paid for your time to write the code to do the job that they want to get done. I think that you are taking advantage of your clients to milk more money out of them by not using modules. If I need to print out a report for a client that wants dates in a certain format, I am not about to waste their or my time rewriting what Date::Calc or DateTime can do faster than I can write the copy of the code for it. Think more about the dead line they have set. Modules will help you meet those dead lines with no problems. I do not know what I would do without List::Util's min and max subroutines. There is also one of my life savers, Data::Dumper that I would not feel like attempting a rewrite.

    I have been told here plenty of times to not rewrite the wheel. That is what it sounds like you are doing here. Learning to use modules in hostile environments that you are describing is probably the best thing that you can do, however, do not do it on your clients' time and dime.

    Have a cookie and a very nice day!
    Lady Aleena
Re: Top 11 (GOOD) reasons not to use someone else's Modules
by scorpio17 (Canon) on Apr 26, 2012 at 14:25 UTC
    Seriously?! It's a little late for "April Fools Day" don't you think?
Re: Top 11 (GOOD) reasons not to use someone else's Modules
by BillKSmith (Monsignor) on May 05, 2012 at 20:32 UTC

    As a side issue, note that there is more than one way to 'use' a module.

    Study of code and/or documentation of related modules often gives me useful insight into a problem, even if I decide not to use any of them.

    Sometimes it is useful to reuse part of the interface. This standardization keeps alive the possiblility of actually using the module in the future.

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by Anonymous Monk on Apr 29, 2012 at 17:23 UTC

    You forgot two:

    1. TROLLING Finished polishing your last script? Have a little time to kill? Feeling ignored? Just hop over to PerlMonks and start a thread about ... oh about anything, really, it doesn't matter, but be sure to include a phrase that matches /not \s+ use \s+ CPAN .* module/xms. Poking a fire ant mound with a stick is about the closest equivalent to the mindless swarming and biting this will provoke.
    2. AD HOC BAYESIAN ADVICE FILTERING Say you value PerlMonks for its wisdom and advice, then you are left with a problem: What advice is good? How can you tell? This problem is a social network analog to that of email spam. Here is a heuristic on how to proceed:
      • Get a pad of paper and a pencil
      • Troll a thread as per the point above, but include an actual valid reason for not using CPAN. Two reasons that strangers are never able to gainsay without some claim to omniscience are, "My company won't allow it," and "I cannot have external dependencies". There are others.
      • Come back to the thread in about a day. Write down the user names of the responses and weight them as you will. Remember, the point is not to mark down people for advocating CPAN, but to mark down people for mindlessly advocating CPAN.
      • Keep this list of weighted user names next to your computer or taped to the laptop case. Refer to it when in doubt about the probable sagacity inherent in a future post.

      For the advanced learner, send a quick email to Paul Graham asking how he'd automate this. Then open a thread asking monks to help translate some lisp code to perl. Be sure to mention that the conversion can't use CPAN modules....

Re: Top 11 (GOOD) reasons not to use someone else's Modules
by sundialsvc4 (Abbot) on Apr 24, 2012 at 21:51 UTC

    Well, this is a first.   Never before have I paid 9,645 people a very sincere compliment, and then be redacted by a stranger.   :-}

    “I meant what I said, and I said what I meant ...”

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://966607]
Approved by jettero
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (4)
As of 2024-03-29 13:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found