http://qs1969.pair.com?node_id=250495

CPAN loving monks,

Perl Monks += TMTOWTDI inspired me to re-post a question that I asked more than a year ago. In addition to following CPAN updates, reading talk slides and the advent calendar, reading other people's top 10 lists is a good way to know which modules are loved.

The original post, Favourite modules March 2002, had my own top 10 in it. It's nice to see how much has changed since then.

So here is my module top 10 for April 2003, again excluding my own modules and pragmata.

  1. B::Deparse
  2. Carp
  3. DBD::SQLite
  4. DBI
  5. File::Find::Rule
  6. IPC::Run
  7. LWP
  8. Regexp::Common
  9. Benchmark
  10. MIME::Lite
And of course CPANPLUS to install them :)

What are your favourite modules?

Juerd
- http://juerd.nl/
- spamcollector_perlmonks@juerd.nl (do not use).

Replies are listed 'Best First'.
Re: Favourite modules April 2003
by mirod (Canon) on Apr 15, 2003 at 08:28 UTC

    Here they are (there are more than 10, but that's why Perl is great: tons of super-useful modules):

    Modules that make Perl what it is:

    • CGI (+ CGI::Carp), I am an old school guy (but I use it with mod_perl when it makes sense though),
    • DBI (+ DBD::mysql, DBD::Pg, DBD::SQLite), SQLite for quick hacks and when there is only one user, Pg most of the time,

    Very useful:

    Convenient modules I use a lot these days:

    • Memoize, especially to memoize the output of the following one,
    • Digest::MD5, to sort through heaps of archived data and figure out what was stored twice under different names

    XML modules:

    • XML::Twig, surprise surprise ;--)
    • XML::Simple, battles with YAML for my configuration storage needs,
    • XML::PYX, for one-liners,
    • XML::LibXML, when I manage to get it installed, which is not that often

      Just a comment/question on what you mentioned here:

      • Memoize, especially to memoize the output of the following one,
      • Digest::MD5, to sort through heaps of archived data and figure out what was stored twice under different names

      I'm just wondering what benefit Memoize provides in this context. It seems from your brief description that you are doing MD5s of the archived data so that you don't have to keep all that data in memory (just an MD5 hash of the data). This will make it easy to find duplicates and won't take up much memory. But my memoizing it, you are still keeping all the archived data in memory, and you are keeping the MD5 hash in memory as well. You might as well just store the data itself in a hash and do a straight comparison on it saving the time required to do an MD5 hash on it.

      I'm curious to know if I am blatantly missing something here, or missunderstanding the usefulness of Memoize in this context.

      By the way, I think Memoize is a great module, but I don't think there are many situations where it is actually beneficial.

        The function I memoize gets passed a file name, slurps the file, normalizes spaces and computes its MD5. So I don't think the content of the file is cached, as it is internal to the function.

        I agree that using Memoize only saves me the cost of a hash (filename => MD5). I just like how easy it is to use it, and how it removes some extra code. As programers we are used to adding extra data structures and code to cache that kind of result, but really, using Memoize gets us closer to the initial algorithm for solving the problem. At least that's how I justify using it here ;--)

      YAML, successor to Data::Denter, nicer output IMHO than Data::Dumper, safer to use too, modified to allow for variables (a-la-XML::Simple in its latest version)

      YAML is not particularly accurate, and the author has no intention of improving matters. Personally I wouldnt use it at all, and I certainly wouldnt recommend it to others without serious caveats.

        I have heard those claims, but the data I deal with is usually quite simple, just hashes/arrays/scalars/simple objects, no code refs or the likes. So it looks accurate enough for me.

        Could you be more precise and give us examples of the kind of data that causes problems with the module?

        As a side note I exchanged a couple of emails with ingy when I wanted to patch the module, and he was most helpfull.

Re: Favourite modules April 2003
by Corion (Patriarch) on Apr 15, 2003 at 08:51 UTC

    My Favourite Modules in use for April 2003

    1. XML::XPath - XPath expressions rock da house for extracting stuff
    2. WWW::Mechanize - Since I do webscraping a lot
    3. CGI::Wiki - Since I use it for my wiki

    My Favourite Modules not in use for April 2003

    1. File::Find::Rule - I recommend it wherever I can, but haven't used it myself yet. But I like the API.
    2. dotdotdot - the yadda yadda yadda operator makes for nice, compiling examples, and still prevents the user from blindly running the example without changing the minor nits.
    3. P6P5 by xmath - Perl 6 rules rule !

    My least favourite module currently in use for April 2003

    1. HTML::Tidy - it's got no documentation, it's got a horrible interface from Perl, is written in C++ and (under Linux) it has a LOT of underdocumented dependencies that can't be installed through my package managers (apt-get resp. CPAN). This is the first module where installation under Windows was easier than under Linux. Watch out for HTML::Tidy::Simple, a wrapper that will try to user HTML::Tidy and, if its not there, shell out to the tidy program (which is what I now use under Linux).
      ptidy sourceforge page (0 released files)
      tidylib Perl bindings (got this to work)

    Update: Added links to HTML::Tidy

    perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web
Re: Favourite modules April 2003
by valdez (Monsignor) on Apr 15, 2003 at 12:31 UTC
Re: Favourite modules April 2003
by AcidHawk (Vicar) on Apr 15, 2003 at 11:57 UTC

    ++ Juerd for a good idea.

    I am currently using a host of modules which I have varying success with. But as I do most of my work on Win32 platforms I use mostly modules that make my life easier in that environment.

  • Win32::Daemon
  • #Manipulate the Win32 Service]
  • Win32::Process
  • #Spawn external process'.
  • Win32::EventLog
  • #Needed to send messages to the NT eventlog
  • Win32::EventLog::Message
  • #Use this and you don't need a Message DLL
  • Win32::API
  • XML::Simple
  • #For the Managed Domain Cust Mapping
  • File::Copy
  • #To Move the XML form the xmlout dir to the relevant callout dir
  • Win32::TieRegistry
  • #Read and Update/Create Win32 Registry Keys
  • DBI
  • DBD::ODBC
  • CGI
  • IO::Socket
  • #Part of Core
  • IO::Select
  • #Part of Core
  • Email::Valid
  • #Validate email addresses for good form as well as DNS checks

    I could go on but theses are really the basis of most of what I do..;)

    -----
    Of all the things I've lost in my life, its my mind I miss the most.
Re: Favourite modules April 2003
by dreadpiratepeter (Priest) on Apr 15, 2003 at 14:06 UTC
    At the moment:
  • Parse::RecDescent - I am an old lex/yacc language designer. The other day I actually had cause to write a little language. I am overwhelmed by how flexible and powerful this module is. All the joy of yacc without all the pain of yacc Thanks Damian!
  • Quantum::Superpositions - Takes a whole class of simple operations that were too complex in Perl and makes them simple again (comparing lists, set operations, etc).
  • DBD::Chart DBIx::Chart - there is nothing like charts to make PHBs happy. And there is nothing like a package with which you can implement them in minutes to make you look good.
  • Data::Locations - makes it simple to build text when you don't necessarily want to build it sequentially, like when building a web page. You can define handles that print into a specific part of your final output. So you could define a skeleton of a web page, have a script handle, a style handle, a body handle, etc. Write the appropriate stuff to the appropriate handle then dump it at the end.
  • Data::Dumper - Couldn't function without it.
  • Date::Calc - All the date manipulation I ever need, in one place.
  • Spreadsheet::ParseExcel Spreadsheet::WriteExcel Allows you to use excel as a data entry mechanism. A great alternative to web forms for really complex data, or for management-type users who a very proficient in excel, but not-so-good elsewhere, Also a great report generation method.
  • CPAN - best mechanism ever for automated installation. It's the life blood of what make Perl sucessful. (Although I hear the CPANPLUS is even better. It's on my short list of things to try).

    -pete
    "Dude, those llamas are pissed!"
      Quantum::Superpositions - Takes a whole class of simple operations that were too complex in Perl and makes them simple again (comparing lists, set operations, etc) - that was a suprise for me. The name does not idicate that it's such a general purpose module.

        I've known about Q::S for a long time, but I always thought it's a joke (and inefficiently implemented) that TheDamian forgot to put inside the Acme namespace :) (Don't know where I got that impression). However, I happily admit that I was wrong -- there is quite a number of articles and nodes about Q::S now that show its usefulness. It was only today that Randal posted a link showing how to win football bets all the time using Q::S :).

        Also, recent versions seem to be rather efficient. I haven't looked at the source yet, but I assume that it's more efficient and better debugged than anything I'd quickly hack together if I were in the need for it.

        A surprise to me as well, and I'll be reviewing it again.

        As dense as I must be, from having read about it earlier, I guess I didn't understand the viewpoint or jargon or whatever from various write-ups, including it's own docs.

        I never thought I was stupid... but maybe I should rethink that...

        Think perl 6's junctions.

      Just a word of caution to anyone seduced by the power of Quantum::Superpositioning (as I was around 10 months ago:) : Quick, it ain't; Memory hungry it is.

      It's a fun, and very powerful demonstraton of some powerful concepts, but using it in workaday code is costly.

      To demonstrate that, I reimplemented the brute force algorithms shown in the Q::S pod for is_prime(), factors() and GCD() using fairly standard perl, and the result show that this is not permature micro-optimisation I am talking about:

      The benchmark

      The results

      D:\Perl\test>qs-test s/iter Q::S perl Q::S 282 -- -98% perl 4.87 5702% -- Primes found: Q::S:1229; Perl:1229 s/iter QS_factors PL_factors QS_factors 322 -- -100% PL_factors 0.120 267905% -- qs:90 80 2 48 180 360 18 72 30 144 16 6 240 120 3 36 40 9 12 15 20 8 4 + 60 24 45 10 5 pl:2 3 4 5 6 8 9 10 12 15 16 18 20 24 30 36 40 45 48 60 72 80 90 120 1 +44 180 240 360 s/iter QS_GCD PL_GCD QS_GCD 336 -- -100% PL_GCD 0.148 226174% -- QS:60 - PL:60

      To put that into perspective, the non-Q::S version of is_prime() will find the 9592 prime < 100,000 in approximately half the time the Q::S version finds the 1229 < 10,000.

      The non Q::S version of factors() will find the approx: 400,000 factors of 2!..9! in around a 5th of the time required by the Q::S version to find the approx 6000 factors of 2! .. 6!.

      And The non-Q::S version will perform the approx: 400,000 divisions and int's and the approx: 800,000 comparisons involved in finding the GCD() (using this algorithm) between each succesive pair of factorial 2! thru 9! in about 1/5th the time taken for the Q::S version to do the (approx) 12,000 divisions, 6,000 int's and 12,000 comparisons.

      Even when used for simple testing of arrays against constants Eg.if (all(@array) != 1) {...) the overheads are quite heavy and can quite quickly invoke swapping if your arrays are of any size.

      Hopefully, once the functionality is moved into the core as C or XS, this caveat will disappear.


      Examine what is said, not who speaks.
      1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
      2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
      3) Any sufficiently advanced technology is indistinguishable from magic.
      Arthur C. Clarke.
Re: Favourite modules April 2003
by PodMaster (Abbot) on Apr 15, 2003 at 11:16 UTC
    I too am leaving out modules I maintain (btw I contribute on a lot of my faves )...
    1. Wx -- aaaaaw yeah!!!!!!!
    2. ExtUtils::Installed -- I use this a LOT
    3. PPM::Make -- this one as well
    4. ExtUtils::ParseXS -- this should be core (this is how xsubpp should be done)
    5. CGI::Wiki::Simple -- not bad, could be better ;)
    6. File::Spec -- portability, damn I like it
    7. Regexp::Common -- beginning to like it a lot
    8. Mail::Box -- it's getting better
    9. Audio::Beep -- fun ;D
    10. Module::Dependency::Grapher -- also fun
    11. CGI::Application -- I still love it ;)
    12. DB_File and/or BerkeleyDB -- and these, but you knew that ;D
    There are a couple of others that could be on my list, but are currently unfinished, so they are not (CPANPLUS for example, or Test::Smoke).


    MJD says you can't just make shit up and expect the computer to know what you mean, retardo!
    I run a Win32 PPM repository for perl 5.6x+5.8x. I take requests.
    ** The Third rule of perl club is a statement of fact: pod is sexy.

      File::Spec -- portability, damn I like it

      Except when it makes your life hell due to incompatability with other modules, as you have to regex everything into Unix style to get modules like File::Basename to work with it. Or try using with a file upload from an OS that isn't the same as the server. As a consequence I now avoid it like the plague.

      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      Barbie
      Birmingham Perl Mongers
      Web Site: http://birmingham.pm.org/
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

        Could you add some more information about the incompatabillity prolem? I thought File::Spec was the way to go and I use it in one of my projects.
        The trouble you seem to be having is because you haven't read the fileparse_set_fstype portion of the File::Basename docs. Anyway, I know its been a year since you wrote this, but I finally wrote fileparse (just for completeness) and thought I'd share.
        sub File::Spec::dirname { my $self = shift; return $self->catpath( ( $self->splitpath(shift) )[ 0, 1 ] ); } sub File::Spec::basename { my $self = shift; my $name = shift; return ($self->fileparse($name, map("\Q$_\E",@_)))[0]; } sub File::Spec::fileparse { my $self = shift; my( $suffix, $path, $name ) = $self->splitpath(shift); $path = $self->catpath( $suffix, $path, '' ); for my $s(@_){ $self =~ s/($s)$//i and last; } $suffix = $1 ? $1 : undef; wantarray ? ( $name, $path, $suffix ) : $name; }
        Check out file.spec.basename.txt to test the above with the File::Basename test suite.

        MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
        I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
        ** The third rule of perl club is a statement of fact: pod is sexy.

        Maybe you'd like Path::Class better. I sure do, and I hope it isn't just because I wrote it. -Ken
Re: Favourite modules April 2003
by hiseldl (Priest) on Apr 15, 2003 at 14:33 UTC

    Here are my favorites:

    • DBI - I do a lot of database work, and a lot of times I just need to see if a DB has a table or is available, so I use 'dbish' a lot. It's also a quick way to figure out what DSN's are available on your machine.
    • CGI - cgi scripting. :-)
    • CGI::Application - more often than not, defining callbacks for actions is a lot easier than straight scripting.
    • HTML::Template - use this with CGI::Application for faster web app prototyping.
    • DBIx::Recordset - has a nice interface.
    • HTML::TreeBuilder - once you wrap your brain around this one, you can scrape data from any web page you want. There's also a module HTML::Seamstress which uses this to create a better HTML template system than HTML::Template (IMHO).
    • Tk - sometimes it's nice to have your favorite script with a GUI interface. :-)
    Update: Here's another one:
    • CGI::Session there's a cookbook and some good tutorials for using this module.
    I use several others but these seem to be the most used in my arsenal, and I am a frequent surfer of CPAN, always watching out for anything that makes my job easier.

    Cheers!

    --
    hiseldl
    What time is it? It's Camel Time!

Re: Favourite modules April 2003
by MrCromeDome (Deacon) on Apr 15, 2003 at 14:06 UTC
    Here's my 10 faves, in no particular order:
    1. HTML::Template - For keeping Perl tidy
    2. DBI - Because I shouldn't have to change my programs because I change databases
    3. XML::Simple - I don't know much about XML, but this has been a help to me
    4. Mail::SpamAssassin - Because I HATE SPAM!
    5. File::Slurp - It frequently helps me save a few lines of code here and there
    6. CGI::Carp - Still too many dumb mistakes
    7. CGI - When it has to be done quick and dirty, accept no subsitute.
    8. Archive::Zip - Makes delivering our applications and other stuff to our clients a breeze
    9. Imager - Tremendously easy to work with images and scanned documents
    10. LWP::UserAgent - Too many uses to list.
    There's more I love and use, but these are the ones that have served me best, from my baby Perl days till now (only slightly better than baby Perl, I'd imagine :P)

    MrCromeDome

Re: Favourite modules April 2003
by crenz (Priest) on Apr 15, 2003 at 09:23 UTC

    Good post, Juerd! I don't have a long list to offer myself. This month is spam-catching month for me... I finally got around writing myself a nice mail filter. My top two modules for this month are

Re: Favourite modules April 2003
by dws (Chancellor) on Apr 15, 2003 at 17:13 UTC
    Top of the List for April, 2003

    I'm finding that

    make a kick-ass combination for adding usage information to scripts, and is letting me throw out a lot of grotty, hand-rolled scraping code.

Re: Favourite modules April 2003
by ignatz (Vicar) on Apr 15, 2003 at 15:48 UTC
Re: Favourite modules April 2003
by Aristotle (Chancellor) on Apr 15, 2003 at 23:52 UTC

    I'm currently totally infatuated with the following:

    The following modules are basic stuff and don't get you far on their own (as opposed with the ones listed above). Still, they're so useful I could not imagine living without them anymore.

    • strict - need I say more?
    • LWP - feels kludgy at times, but rock solid and reliable, nevertheless
    • CGI - dito
    • Getopt::Long - turning q&d tenliners into flexible tools amazingly quickly
    • File::Spec::Functions - symbolic manipulation makes for readability and portability
    • Data::Dumper - Words cannot do this module justice. I've used it in so many different situations, from communication to debugging to logging to persistence, that it alone deserves as much praise as all other modules together. And I'm not sure that'd be doing it justice. Along with it, Safe, while not used much, deserves a mention.

    Besides those, Mail::SpamAssassin and Perl::Tidy have earned an honorary mention for doing good work on my system, though I don't use them directly. I like Cache::Cache and friends, but don't need those often. Modules that might have made my "infatuated" list, had I had much opportunity to use them, include Log::Log4perl, WWW::Mechanize, DBD::SQLite, URI::Find, Inline::TT, Tie::File, Inline::File, CGI::Application, Email::Valid, and more.

    Makeshifts last the longest.

Re: Favourite modules April 2003
by hsmyers (Canon) on Apr 15, 2003 at 21:57 UTC

    Here, including my own stuff are my top 20 according to useage---

    • 0010, Carp::Assert
    • 0011, XML::Twig
    • 0012, Chess::PGN::Moves
    • 0013, Carp
    • 0013, Wild
    • 0016, Chess::PGN::Filter
    • 0016, Getopt::Std
    • 0017, Parse::RecDescent
    • 0018, File::Basename
    • 0021, DB_File
    • 0021, Pod::Usage
    • 0022, Text::Wrap
    • 0023, Getopt::Long
    • 0027, Text::DelimMatch
    • 0032, Text::CSV
    • 0039, Chess::PGN::Parse
    • 0045, Test
    • 0046, ExtUtils::MakeMaker
    • 0048, Data::Dumper
    • 0062, Chess::PGN::EPD
    • 0068, constant
    • 0223, diagnostics
    • 0223, warnings
    • 0239, strict

    How do I know this? Fairly easy, go to the root of your Perl development tree on the box of your choice and append the results of 'grep -rd "^use " *.pl' to this:

    #!/perl/bin/perl # # use.pl -- statistics for 'use' module name. use strict; use warnings; use diagnostics; my %module; my @list; while (<DATA>) { if (/^use\s(.*?);/) { my $what = $1; if ($what =~ /^lib/) { $module{'lib'}++; } elsif ($what =~ /^constant/) { $module{'constant'}++; } else { $module{$what}++; } } } for (keys %module) { push(@list,sprintf("%04d|%s",$module{$_},$_)); } for (sort @list) { my($howmany,$what) = split /\|/; print "$howmany, $what\n"; } __DATA__

    And then run the script...

    --hsm

    "Never try to teach a pig to sing...it wastes your time and it annoys the pig."
      You can certainly do alot better than that long piece of code for this task though. The last two loops can be combined:
      for (sort { $module{$a} <=> $module{$b} } keys %module) { printf "%04d, %s\n", $module{$_}, $_; }
      Gets you rid of the synthetic @list too. I have no idea what your if/elsif are supposed to sort out, either. And finally, instead of using <DATA>, you can use <> and just pipe the output from grep to the script. A compressed version:
      /^use\s+([^\s;]+)/ && $_{$1}++ while <>; printf "%04d, $_\n", $_{$_} for sort { $_{$b} <=> $_{$a} } keys %_;
      And then it's just
      $ grep -hr ^use perl/ | perl foo.pl

      Makeshifts last the longest.

      Does usage though == favourite module? (Thanks for the code snip it though!)

      ----
      Zak
      Pluralitas non est ponenda sine neccesitate - mysql's philosphy
Re: Favourite modules April 2003
by Solo (Deacon) on Apr 15, 2003 at 21:17 UTC
Re: Favourite modules April 2003
by toma (Vicar) on Apr 16, 2003 at 04:48 UTC
Re: Favourite modules April 2003
by l2kashe (Deacon) on Apr 16, 2003 at 19:59 UTC
    From a little old unix admin:

    IO::Socket
    FileHandle
    File::Find
    DBI ( et all )
    Getopt::Std
    SNMP, Net::SNMP, SNMP::Session (*ick* but necessary)
    Net::(Ldap | SSH | SCP | FTP | Telnet)

    Date::Manip
    I think that about sums it up for the admin side..On the reporting side

    CGI ( though I almost loath HTML, manager types like it)
    GD::*, as well as the chart family and 3D stuff in there as well



    MMMMM... Chocolaty Perl Goodness.....
Re: Favourite modules April 2003
by zakzebrowski (Curate) on Apr 16, 2003 at 18:39 UTC
    No one mentioned Date::Manip so I'll add that to the list...

    ----
    Zak
    Pluralitas non est ponenda sine neccesitate - mysql's philosphy
Re: Favourite modules April 2003
by m-rau (Scribe) on Feb 10, 2005 at 09:26 UTC
    Currently my favorite modules are
  • XML::Smart
  • DBI
    Currently my standard modules are
  • Getopt::Long
  • Pod::Usage
  • Config::IniFiles
  • File::MkTemp
    Currently I am pretty excited about
  • POE and especially
  • POE::IKC::Server
  • POE::IKC::Client
  • POE::Session
    The thing with the POE modules is, that I am really a little bit confused about the following thoughts in my mind
  • will these modules be replaced with a better framework?
  • is there still activity in the enhancements of the modules (especially IKC)
    Any thoughts about these questions are very welcome. What are your experiences with POE?