hermida has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

Question of interest, just trying to understand why in other languages like Python for example are very different when installing libraries, particularly not going through any test suite and for libraries that have integrated C/C++ code it doesn't have to compile it for your installation when you install.

For example I am using a popular bioinformatics application written in Python (it should have been written in Perl but some bioinformaticians and software developers in the computational biology/bioinformatics/biomedical research community I'm part of decided not to use Perl for some large projects (for reasons that were mostly out of lack of knowledge of what Perl can do) when most every library and app we had before was in Perl and also Java... sorry off topic)

When you first configure the application it automatically fetches library dependencies i.e. Python "eggs" for your version of Python. I see there are multiple libraries that I know have integrated C code and it just fetches them and puts the eggss in the desired folder with no compilation or testing whatsoever. For example, NumPy which has a lot of integrated C code and also pysqlite which has C code it just downloads the egg and I double-checked its not compiling absolutely anything.

You just see this:

Fetched http://eggs.g2.bx.psu.edu/new/numpy/numpy-1.3.0-py2.6-linux-x8 +6_64-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/new/pysqlite/pysqlite-2.5.6_3.6.17_s +tatic-py2.6-linux-x86_64-ucs2.egg

So, finally the question: why does Perl need to go through all this extra rigmarole of testing and compiling any library C/C++ code?

  • Comment on Why are other popular languages very different from Perl when installing libraries, e.g. no testing needed and no compilation of C/C++ code done
  • Download Code

Replies are listed 'Best First'.
Re: Why is it in some other popular languages fewer steps and potential issues when installing libraries no testing needed and no compilation of C/C++ code done
by Corion (Patriarch) on Apr 06, 2011 at 13:14 UTC

    Perl has a long-standing tradition of testing. You can always choose to ignore the test results or just disable testing when installing modules.

    Out of personal experience though, I find that the tests run mostly reflect the assumptions of the programmer writing the module. So if a module test fails, this means, at least in my personal experience, that at least one assumption of the original programmer fails. I prefer an early notice of such failing assumptions over code that just fails while I try to figure out what the code does.

      But what about not needing to compile any C/C++ code that is in the library?

      Just trying to understand to see how Perl can be made easier for beginners and not-so-beginners. People like us who have used the language for a long while and know well CPAN/CPANPLUS/App::cpanminus have no issues, but we should make things as easy as I see it in Python, beginners are the people that become our future Perl community and expand the usage of the language.

Re: Why is it in some other popular languages fewer steps and potential issues when installing libraries no testing needed and no compilation of C/C++ code done
by cavac (Prior) on Apr 06, 2011 at 16:14 UTC

    So, finally the question: why does Perl need to go through all this extra rigmarole of testing and compiling any library C/C++ code?

    Because most Open Source projects (and quite a lot closed source ones as well) like to change API's seemingly just because they can (the typical "f.ck backward compatibility..." issue).

    Also, the user might use a different compiler, may want to link against a patched/alternative library, or may want to use an operating system not supported by the binary build.

    I also like to comment on this: It's far easier (read: takes much less time and effort) to provide binary packages for Windows (only have to do a x86 and x64 build) than it is for the many different Linux distributions on multiple hardware architectures. And there's also the BSD systems, commercial Unix systems, OSX and whatnot. Most of them releasing a new version once or twice a year. Personally, i'd rather have module authors (like myself) write better modules and some selftests for when things go wrong than have them spend all their time struggling with different operating systems - if there are problems, in many cases the automated tests can report back to the author via tools like cpantesters.org.

    Don't use '#ff0000':
    use Acme::AutoColor; my $redcolor = RED();
    All colors subject to change without notice.
Re: Why is it in some other popular languages fewer steps and potential issues when installing libraries no testing needed and no compilation of C/C++ code done
by Anonymous Monk on Apr 06, 2011 at 13:24 UTC
    So, finally the question: why does Perl need to go through all this extra rigmarole of testing and compiling any library C/C++ code?

    It really doesn't, both cpan and cpanp allow you to skip running tests

    ActiveState provides eggs, calls them ppms, with a tool called ppm.

    Ubuntu and the like also provide binary packages via apt-get.

    cygwin also provides binary packages , as does MinGW (mingw-get)

    There are other binary packages like StrawberryPerl or CitrusPerl or IndigoPerl, they also support ppm.

    Even Padre, the Perl IDE, comes as a binary package.

      Oh yes thank you for reminding me, I should have remembered that one can get rpms, apt packages and other means to get binary packages.

      Just throwing out an idea: is there a way in Perl like I have described in Python to, in a universal way, create binary packages of CPAN distros yourself and package them into one file and that these can be used by Perl without having to unpackage the file? The ones listed are all very particular to different types of OS. Maybe this should be invented for Perl to make things easier for beginners and dependency management. Maybe PAR?

Re: Why is it in some other popular languages fewer steps and potential issues when installing libraries no testing needed and no compilation of C/C++ code done
by ikegami (Patriarch) on Apr 06, 2011 at 16:26 UTC

    ActiveState and others provides binaries via ppm, primarily for Windows. Unix distros provide binaries via their package system. You can do quite a bit without a compiler and without testing.

    But say you do need a compiler, perhaps because you want a module that hasn't been tested and packaged for you yet. Perl provides a very easy installation mechanism in the form of cpan. On Windows, it will even automatically install a compiler for you if you need one!

    I can't imagine a more refined system.

      Yes, but what cavac just said is entirely correct:   Windows is a much smaller target to hit, in a way, than Linux and Unix ... which might be deployed on anything from IBM MVS/XA Z/OS to a smartphone.   It’s also worth noting that a language compiler is usually not available by default on Windows boxes.   The ActiveState Perl (vs. Strawberry) team takes a rather unique approach to their deployments, and of course it is a purposeful part of their chosen strategy that they have done so.   (As Strawberry’s approach is an equally purposeful part of theirs.)

        Windows is a much smaller target to hit, in a way, than Linux and Unix ...

        And? You forgot to explain how you think this relates.

        It’s also worth noting that a language compiler is usually not available by default on Windows boxes.

        And? Again, you forgot to explain why you think this matters.

        The ActiveState Perl (vs. Strawberry) team takes a rather unique approach to their deployments, and of course it is a purposeful part of their chosen strategy that they have done so.

        You forgot to mention what you're talking about. I think you're referring to ppm or its installation of MinGW, but you're wrong either way. Neither are unique to ActivePerl.


        You seem to have rushed your post, posting only premises and hoping we'd be able to guess the arguments you would form from those premises. Please start over.

        A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Why are other popular languages very different from Perl when installing libraries, e.g. no testing needed and no compilation of C/C++ code done
by locked_user sundialsvc4 (Abbot) on Apr 06, 2011 at 17:02 UTC

    I don’t have any beef with the folks who develop other languages – having used most of them.   I am very impressed, however, with the almost fanatical emphasis that the Perl community places on thoroughness and testing ... and the way that it is tightly integrated into the cpan engine.

    And ... we don’t call ’em by cutesy names, like “eggs” or “gems” or “beans.”   :-D

    The teams that work on “those other” languages are damm-good engineers too.   It would be wrong to say that their work is somehow lesser, because it plainly isn’t.   But I am surprised that they don’t seem to religiously copy this very admirable trait of Perl’s deployment strategies.

Re: Why are other popular languages very different from Perl when installing libraries, e.g. no testing needed and no compilation of C/C++ code done
by cdarke (Prior) on Apr 07, 2011 at 10:55 UTC
    As someone who has written XS code with Perl and used the Python C API to create modules, I have to say that the Python mechanisms seem much slicker. Python's distutils is so easy to use, I really wish there was a Perl equivalent ("so write one": if only I had the time). You could argue that ExtUtils::MakeMaker is an equivalent, but that is a generation behind distutils.

    As usual I have learnt a lot from the comments on this node from others. I wonder if one of the reasons why the Perl module installation process is so unfriendly might simply be that there is a lack of knowledge of the tools available (or maybe that's just me?). There are so many ways to do it in Perl, but most just want the best way, and need that to be in the main documentation and be accessible. OK, so we have perlnewmod and its ilk, and that gives us the kind of user experience that hermida is complaining about.

    Right now a Perl author has to look hard for the best way to create a distribution, with Python you are given it.

      I would have to agree with cdarke in terms of the area of the Perl language involving integrating C/C++ code. As a person who just started last year learning how to do it from scratch I can tell you compared to other parts of the language there is a lot of room for improvement, it's just more of a pain in the neck than it needs to be. If you haven't already read chromatic's blog entry on part of this topic Less XS/More CTypes. And a lot of this difficulty brings me to a mediation I made about Putting Perl Back on Top in the Fields of Scientific and Financial Computing, integrating C/C++ code efficiently is a core requirement in these fields and guess what currently Python is taking over Perl's spot because it is much easier for people to do that in Python.

      In terms of creating modules and distributions I think Perl has many wonderful tools, particularly Dist::Zilla and for building and installing more modern replacements for ExtUtils::MakeMaker which are Module::Build and Module::Install.

      In a general sense going on what cdarke said I think there is a lot of work being done and still needs to be done to vastly lower the learning curve it takes for a brand new person to come into the Perl language and most importantly stay there. This is the community area when I have time I try to contribute to because I think it has big bang for the buck. On this one point I have to agree that Python has been doing a somewhat better job, making it so easy to find how to do things and then to do them. You know how new people are if things get too frustrating then bye bye they go to some other language. There is nothing more important than people if you don't have people you don't have a language. People who know Perl then train other people and it just snowballs from there. The old masters will retire one day and we need new, bright and innovative people coming in to keep the language thriving, evolving, dynamic, etc... you know what I mean it's been said millions of times before.

      Right now a Perl author has to look hard for the best way to create a distribution, with Python you are given it.

      Horseshit

Re: Why are other popular languages very different from Perl when installing libraries, e.g. no testing needed and no compilation of C/C++ code done
by JavaFan (Canon) on Apr 06, 2011 at 16:30 UTC
    [About Python] particularly it doesn't need to run through any test suite
    That's because Python is a much better language than Perl. All good coders program in Python, while the evil ones pick Perl. So, programs written in Python are bug-free by design (due to the excellent choices enforced by the mighty Guido), while programs written in Perl are mostly line-noise. Revisiting any part of your program makes it very likely that you will introduce a bug (one of the reasons for that is that Perl does too much behind your back). Hence all the tests coming with Perl modules. Not that it actually improves the quality of the delivered code - it's written by the same evil coder that wrote the module in the first place.

    The only way to properly test a Perl module is to have a test script written in Python. But then it'll take so long to fix all the bugs, the entire thingy might as well have been written in the Holy Language in the first place.

      You're being sarcastic right???

      Every post I see from you all over the place is just slinging this bullshit why the do you bother being here? Oh yes I read "penance", I would really love to know for which sins you desperately desire to be forgiven? Please ignore this if you really are being sarcastic...

        I would really love to know for which sins you desperately desire to be forgiven?
        I once smiled at a Python conference.

        Guido told me to program in Perl for 10 years - after which I may ask for forgiveness. Only 7 more years to go.

        Yes, he is being sarcastic. Nearly all the time. But he also gives good advice sometimes, and if you like sarcastic humour (a vice of mine), he often is quite funny. I realize that I'm probably in the minority with this view, but I can't help it

        Outing myself as

        JavaFanFan

Re: Why are other popular languages very different from Perl when installing libraries, e.g. no testing needed and no compilation of C/C++ code done
by locked_user sundialsvc4 (Abbot) on Apr 07, 2011 at 17:22 UTC

    I agree with the notion ... “so, go write one.”   And by that I mean that all such efforts will be warmly welcomed (and kibbutzed mercilessly, “peer review”).

    There are lots of situations where a binary can be distributed, and some where you basically need one.   In the Windows environment, for instance, there likely will not be a C/C++ compiler installed on the target system.   The core supporting-libraries come from a single, known vendor:   Microsoft.   All of the linking that is going to take place is going to be dynamic linking, using (AFAIK) only stdcall.   You might have to probe for 32/64-bit differences but that’s about it.

    The technique of compiling and linking the packages in source-code form, on the target system, is similar to the approach used by the Gentoo (and Linux From Scratch) distributions.   Configuration scripts and the compiler make all the decisions.   The target system could be anything from a mainframe to a smart-phone and, provided that the toolchain is properly configured, it will Do The Right Thing.

    Which is “right,” “wrong,” “better,” “worse?”   I would argue that it really does not matter too much, as long as, when the dust settles, you have Done The Right Thing ... extremely reliably.

    Personal Footnote:

    My cursory comments about ActiveState vs. Strawberry, BTW, were not intended to dredge up any sort of “discussion” about them, nor their particular approaches to enabling users who are thoroughly used to “installers” to successfully deploy Perl on their Windows machines.   I regret that some people saw it that way, and were quick to the perceived “challenge.”   If I am utterly wrong about some fine-point of what either or both teams did, it will not be the first time, nor the last.

    I also never intended to slam any language, nor even to compare them.   Designers and Implementors, of languages and of operating systems – and those who build deployments – are among the very best software engineers in the world.   The literally millions of deployments of their work, in mission-critical situations, speaks all that need be said about their prowess.