http://qs1969.pair.com?node_id=1154914

bt101 has asked for the wisdom of the Perl Monks concerning the following question:

Hi I normally write one-file scripts. However I have written a rather extensive set of routines and I would like to re-use the code for several programs. I gather the way to do this is to split up the code and put it into packages. However my mind is boggling on how several things should work. One example is...let's say your original one-file program writes logs using log4perl. In that case you simply initialize the log and then call routines to write to that log from anywhere in your program. How does/should this work when you have split the code into packages? If you initialize log4perl in the main code, how to all of the package routines write to the log file? It is so simple with C where multiple files are just linked into one flat program. I'm fundamentally missing something.
  • Comment on Proper way to create packages and re-usable code?

Replies are listed 'Best First'.
Re: Proper way to create packages and re-usable code?
by Discipulus (Canon) on Feb 11, 2016 at 09:23 UTC
    Hello i'm less or more in the same situation. The moment when you pass from standalone scripts to modules is crucial (from my point of view) moment that tells the scriptor from the progammer.

    You abstracts functionalities (possibly grouping them on affinities) from your main script that become a bare bone sketch of your desired behaviour of the application.

    So if your application is ZXappz and you need to log and also use a database connection to read data you'll probably need a ZXappz::Logger and a ZXappz::Database modules. maybe the main ZXappz.pm just loads these two modules. Is ZXappz::Logger that uses log4perl and ZXappz::Database will load DBI

    This in the simplest case ie when you just group functionalities into modules for a logical separation. The application can be one or more scripts that use such functionalities (subroutines exported by your modules) to do the work.

    In this perpective you gained in readability and maintenence. But there is a better gain: you can document and test your software in a just one place: the module file. Better: you can embrass the "test driven" develop method that will save your time in the long run. You imagine a functionality let's say you want to write to alocal file whe the database is offline. Ok, first you choose a meaningful name like db_fallback_write and you imediatly spot ZXappz::Database as the right place to put it. Then, before coding the subroutine, you wite the POD documentation in the module for the functionality. you describe it as you wont it. Then you write an initial test (a .t file in t/ dir of the ZXappz::Database distribution) for db_fallback_write let'say return the name of the file written on success and 0 on failure. only now you start to code the sub db_fallback_write in the ZXappz/Database.pm file. You are defining an interface for your software: the db_fallback_write functionality want some parameters and returns something. it does it's work as described in the POD.

    db_fallback_write may needs to chose a filename based on the day of the week. This is something private to the sub implementation: the final user of db_fallback_write (you or your script) does not need to know how the filename is composed. So in the ZXappz/Database.pm you can have a choose_filename sub (or as frequently happen a name starting with underscore _choose_filename ). choose_filename can have it's own tests but not POD documentation because it is not part of the interface but only of the implementation.

    i hope i gave you the idea. wiser monks can add and specify about the matter. In my homenode you can search for the string About new modules to see a bounch of links on the subject.

    HtH

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      A little related commentary

      Is ZXappz::Logger that uses log4perl and ZXappz::Database will load DBI

      :) That is one of those weird things that don't quite make sense

      log4perl and DBI both provide good APIs that they don't need generic wrappers

      Now if you're talking a static "Config" module or a "Model" module, that makes sense, but not a generic logger/database, log4per/DBI are already that generic module, just configure them and you're done

      Yeah, sometimes it makes sense to have a generic logger/database module as part of your API, but I'd say most of the time it doesn't :) its weird :D

        Ok, stated my limited experience, i just only to say that probably you need ZXappz::Database anyway because it can contains for example store_new_user and store_action_for_user and so on. So in the main code you can just:
        my $appdb = ZXappz::Database->new(); $appdb->store_new_user($previously_generated_name,time); # uses DBI to + insert into a table .. $appdb->close # call the DBI's close but maybe also log somewhere the +succesfull operation

        L*

        There are no rules, there are no thumbs..
        Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
      WOW, thanks everyone for the great responses. I understand what you are saying... Packages should have a narrow purpose and be self contained. I think I see the error of my ways as the code that I want to re-use does not really fit that mold. Basically I have a bunch of routines that do low level serial communication. All of the routines form a whole program that does the communication, error handling, logging etc. At the top level, there is one routine to call to send/receive a message. I then want to use this code for (let's say) 10 different programs where each program only differs by the main loop which loops and calls the re-usable routines for communication. All I "really" want is to house the re-usable code in a separate "file" so there is only one set of re-usable code to maintain. From what I read on the internets, I gathered that the way to split perl into different files was with a package. However my code doesn't really fit the mold of having singular purpose. I'll do some research on the links provided. I understand that if the whole program was destined to be a 1 million line ERP system, the it would be worth re-organizing and rewriting everything to fit into modules. However I'm aiming at just splitting things into "two" files, and I'm 1 yard from the finish line. So cut/paste into 10 scripts may be the way to go.

        The following is not the proper way to modularize your code, but because you are just 'one yard before the finish line' you could use a script or Makefile to concatenate multiple files to a single program. That is easier and less error prone then the temporary "manually cut and paste" solution, that you are considering.

        Assuming you have the following files:

        -rw-r--r-- 1 hadri hadri 51 Feb 12 04:24 main_barack -rw-r--r-- 1 hadri hadri 63 Feb 12 04:15 my-serial-library.src -rw-r--r-- 1 hadri hadri 68 Feb 12 04:27 postlude_general.src

        Your general libary routines are in my-serial-library.src and postlude_general.src. The 'main' program is the only customized part and is named after your client or location: main_barack.

        With a simple Makefile you can generate your program serial_com_barak.pl:

        $ make cat my-serial-library.src main_barack postlude_general.src > serial_co +mm_barack.pl

        For another customer you create a new customized main and run 'make' again:

        $ echo "This is for Bernie" > main_bernie $ make CLIENT=bernie cat my-serial-library.src main_bernie postlude_general.src > serial_co +mm_bernie.pl

        The Makefile:

        # -- sample Makefile CLIENT = barack NAME = serial_comm PROG_NAME = ${NAME}_${CLIENT}.pl PRELUDE = my-serial-library.src MAIN = main_${CLIENT} POSTLUDE = postlude_general.src TAR_NAME = serial-comm_${CLIENT}.tgz TAR_FILES = Makefile TAR_FILES += ${PRELUDE} ${MAIN} ${POSTLUDE} ${PROG_NAME}: ${PRELUDE} ${MAIN} ${POSTLUDE} cat ${PRELUDE} ${MAIN} ${POSTLUDE} > ${PROG_NAME} clean: rm -rf ${PROG_NAME} archive: tar cvzf ${TAR_NAME} ${TAR_FILES} # --- end of Makefile

        Note: Make sure that you have a tab character (\x09, CNTRL-I or ^I) before the cat and other commands in the Makefile.Verify with 'cat -t Makefile':

        ${PROG_NAME}: ${PRELUDE} ${MAIN} ${POSTLUDE} ^Icat ${PRELUDE} ${MAIN} ${POSTLUDE} > ${PROG_NAME}

        I understand that if the whole program was destined to be a 1 million line ERP system, the it would be worth re-organizing and rewriting everything to fit into modules. However I'm aiming at just splitting things into "two" files, and I'm 1 yard from the finish line. So cut/paste into 10 scripts may be the way to go.

        No, copy/paste isn't the answer, thats like first week of intro to programming :)

        modules are the answer, modules that are accompanied with tests, your code should live in modules :)

        Maybe

        Something like that, draw a few outlines like that on a piece of paper, putting different stuff in different places, see what makes more sense

        You can see all kinds of examples on cpan, see App::

        https://metacpan.org/release/Module-Starter

        tracker/App::TimeTracker

        booklist/App::Booklist

Re: Proper way to create packages and re-usable code?
by poj (Abbot) on Feb 11, 2016 at 09:45 UTC
    How does/should this work when you have split the code into packages?

    That's explained in Log::Log4perl

    .. you can retrieve logger objects anywhere in the code. Note that there's no need to carry any logger references around with your functions and methods. You can get a logger anytime via a singleton mechanism:
    #!/usr/bin/perl use strict; package MyModule; use Log::Log4perl; my $log = Log::Log4perl->get_logger("MyModule"); sub doit { $log->info("I'm doing it !"); } package main; use Log::Log4perl ':easy'; Log::Log4perl->easy_init(); $log->info("Asking doit"); MyModule->doit();
    poj
Re: Proper way to create packages and re-usable code?
by goldenblue (Acolyte) on Feb 11, 2016 at 08:37 UTC

    Hi bt101,

    maybe it's me who is missing something... but isn't one of the main reasons for having modules in the first place the re-useability in other code?

    So in your example, unless you import that log4perl into every module you write, you create a dependency to your main file. I would not want to do that. If I use a module, I expect it to come with everything it needs, no matter which Programming language I am writing in.

    So my answer would be: be clear in your code and 'use' every package you package needs.

    Don't mix up code snippets with modules. Because to me it sounds like you are just creating code snippets that can't take care of themselves!

Re: Proper way to create packages and re-usable code?
by Laurent_R (Canon) on Feb 11, 2016 at 11:34 UTC
    Although there are many exceptions, I would say that, most of the time, you don't want the functions of your module to have side effects (such as printing to a file). You want these functions to receive some parameters and to return a result to be used by the caller.

    Of course, one obvious exception is if you want to write a logging module, i.e. a module whose aim is precisely to print things out to one or several files. But then, you have to make it generic if you want to re-use it. So either you have to pass back and forth the file handles between the module and the calling program, or you have to store it in such a way that the module will know where to print, which you can do, for example, with closures or with objects.

    Feel free to do that for your own training if you wish, but remember that such modules have already been written, thoroughly tested and made available for free, such as Log::Log4perl. That's probably what you should use for solving a real life problem.

Re: Proper way to create packages and re-usable code?
by apadamson (Initiate) on Feb 11, 2016 at 08:24 UTC

    Hello, I think you have to distinguish in your mind between having a multi source-file program (as you would often do in C) and writing a library of functions (as in libc). If you can imagine your routines as in a library, then can put them quite easily into a Perl module. The point is that library routines are self-contained. You don't, for example, consider printf as part of your program; it is just a library routine you use. In Perl, this is a potential problem with the scope of objects as variables in a module are not local to that module! You can also use "our" to make them very visible. I don't know if this helps but I have written several modules because I believe in re-usable code.

Re: Proper way to create packages and re-usable code?
by Anonymous Monk on Feb 11, 2016 at 08:20 UTC
Re: Proper way to create packages and re-usable code?
by Anonymous Monk on Feb 11, 2016 at 13:01 UTC
    It is so simple with C where multiple files are just linked into one flat program.
    Even in C it's not at all simple, considering shared and dynamic libraries.
    I'm fundamentally missing something.
    I'm surprized that no one mentioned perlmod. I'd recommend to read it, and ask questions if something appears unclear.
Re: Proper way to create packages and re-usable code?
by karlgoethebier (Abbot) on Feb 15, 2016 at 20:33 UTC

    Some thoughts in a hurry: perhaps it might be a good idea to take a look at Log::Any?

    From the friendly manual:

    Getting a logger

    The most convenient way to get a logger in your module is:
    use Log::Any qw($log); our $log = Log::Any->get_logger;

    Please see ibidem for details.

    IMHO Log4j is fubar - as well as Log::Log4perl is.

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Re: Proper way to create packages and re-usable code?
by perlfan (Vicar) on Feb 15, 2016 at 20:29 UTC
    Often times, modulinos will allow one to "ride the fence" between scripts and reusable libraries.