Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

(This is cross-posted on devshed and perlguru due to not getting much answers)

This is a similar question to my previous one about modules but a more overhead question. The example is somewhat simplified.

Say you have a bunch of scripts like request.pl, modify.pl, transfer.pl, pending.pl etc.....
they all contain multiple subs where it reads a directory or contents of a comma separated file and will check for a variety of conditions so it can go on to modify or transfer or view etc.

Right now every script and alot of subs in them re-read the file or files in question, so alot of scripts have something like this, using a foreach, sometimes a while...etc

foreach my $file (@files) { chomp $file; my $filename = "$file"; (open(my $IF, '<', $filename)); my @entry = <$IF>; $lastentry = $entry[-1]; close $IF; chomp $lastentry; my @record = split(':', $lastentry); my $id = $record[0]; my $name = $record[1]; my $street = $record[2]; my $city = $record[3]; my $state = $record[4]; my $status = $record[5]; if (some condition like $status is active)( do something..... )

So I thought instead of all this re-reading, what if a module was called to just read the files once and do different things in different scripts. Read the files, split up the commas in to variables or something and have it in an array or hash....and then based on conditions do something

Example: in the modify script read the files and determine which are in status active and do stuff, and in move.pl determine if city is something specific and do something else with it but not have to re-read the file in each applicable sub and script. I'm sorta missing something because in all cases the file being read is in a while or for loop so what do I do? Just return the array from the module and split in each file or what? Does this make sense?

How would you do this?

---Iconx
  • Comment on Discussion(maybe?) How would you do this? Modules and reading a file/dir
  • Download Code

Replies are listed 'Best First'.
Re: Discussion(maybe?) How would you do this? Modules and reading a file/dir
by davido (Cardinal) on Feb 25, 2014 at 17:55 UTC

    I don't think you've really given us enough to chew. My advice is to just get started with whatever strategy you can come up with; keep it as simple and straight-forward as you can. If that doesn't work out, I doubt you would need more than 100 lines of code before deciding to modify the approach. And if it does work out, you're done without a lot of fretting.

    my %actions = ( greet => sub { print "Hello world!\n"; }, growl => sub { print "Rawwwrrrr!\n"; }, depart => sub { print "Goodbye!\n"; }, unrecognized => sub { warn "I don't know how to $_[0].\n"; }, ); while( <DATA> ) { chomp; if( exists $actions{$_} ) { $actions{$_}->(); } else { $actions{unrecognized}->($_); } } __DATA__ greet growl whine depart

    ...produces...

    Hello world! Rawwwrrrr! I don't know how to whine. Goodbye!

    Without a better understanding of what you're after, all I can provide is this sort of framework for your consideration.


    Dave

Re: Discussion(maybe?) How would you do this? Modules and reading a file/dir
by kcott (Archbishop) on Feb 25, 2014 at 18:47 UTC
    "This is a similar question to my previous one ..."

    We get lots of posts by Anonymous Monk: I've no idea which was your last one. A link would have helped (see What shortcuts can I use for linking to other information?); being logged in would also have helped. Hopefully, I'm not missing important information.

    On to your question. If I knew a little more about your files and what processing each of those scripts was doing, I might have provided a different (better) suggestion; however, the following should at least provide some guidance.

    Here's a module that's performs all the file parsing in one place. An appropriate handler is specified by the script using this module.

    package PM::SharedFileTasks; use strict; use warnings; use autodie; my %handler_for = ( modify => \&handle_modify, move => \&handle_move, ); sub handle { my ($function, $files) = @_; for my $file (@$files) { open my $fh, '<', $file; chomp(my @record = split /:/ => (<$fh>)[-1]); print "Function: '$function'; File: '$file'\n"; $handler_for{$function}->(\@record); close $fh; } } sub handle_modify { my $status = shift->[5]; # Do something with $status, e.g. print "$status\n"; return; } sub handle_move { my $city = shift->[3]; # Do something with $city, e.g. print "$city\n"; return; } 1;

    Here's how the different scripts might use this module:

    #!/usr/bin/env perl use strict; use warnings; use PM::SharedFileTasks; { # In modify.pl my @files = qw{pm_1076160_1.txt pm_1076160_2.txt}; PM::SharedFileTasks::handle(modify => \@files); } { # In move.pl my @files = qw{pm_1076160_2.txt pm_1076160_3.txt}; PM::SharedFileTasks::handle(move => \@files); }

    Given these files:

    $ cat pm_1076160_1.txt Record 1 ... Record n-1 ID1:Name1:Street1:City1:State1:Status1 $ cat pm_1076160_2.txt Record 1 ... Record n-1 ID2:Name2:Street2:City2:State2:Status2 $ cat pm_1076160_3.txt Record 1 ... Record n-1 ID3:Name3:Street3:City3:State3:Status3

    Here's the output:

    Function: 'modify'; File: 'pm_1076160_1.txt' Status1 Function: 'modify'; File: 'pm_1076160_2.txt' Status2 Function: 'move'; File: 'pm_1076160_2.txt' City2 Function: 'move'; File: 'pm_1076160_3.txt' City3

    -- Ken

Re: Discussion(maybe?) How would you do this? Modules and reading a file/dir
by atcroft (Abbot) on Feb 25, 2014 at 17:35 UTC

    From the example you specified, it sounds like the thing you are missing is the idea of callbacks. I haven't used them much personally (beyond sort()), but it sounds like the idea you want-passing in a reference to a function to be used (a coderef) on the data you are processing. I would recommend looking at Mark J. Dominus's Higher Order Perl or a similar reference for more details.

    Hope that helps.

    Update: 2014-02-25
    Fix error in link to sort().

Re: Discussion(maybe?) How would you do this? Modules and reading a file/dir
by graff (Chancellor) on Feb 26, 2014 at 00:56 UTC
    As others have said, the "problem" is too vague. But just in case it actually involves handling data files that serve specific, predictable, pre-established functions -- such as "address.list", "invoice.table", "personnel.file", "product.inventory", or stuff like that -- a decent approach would be to create a separate module for each type of data.

    Each module knows how to read, modify and save it's own type of data, and when a script uses a module and invokes one of it's methods, the associated data file is read just once, and can be used, updated and saved back to disk as often as you want during the lifetime of any one process.

    If there are multiple steps in a given process that use a given type of file, the corresponding module will always have its current state and content already loaded as module-internal data, with appropriate "getters" and "setters" (and other convenience functions as you see fit) tailored to that type of file.

Re: Discussion(maybe?) How would you do this? Modules and reading a file/dir
by Laurent_R (Canon) on Feb 25, 2014 at 17:48 UTC
    Your question is too general, it is very difficult to figure out exactly what you are looking for or what could be used to solve your issue. It may be that you need simple modules, or possibly callbacks as suggested earlier by atcroft, or possibly dispatch tables, or function factories, closures, objects, etc. Maybe you should give a limited example of something you are trying to do and feel it could possibly be done in a more optimal way (e.g. with less code duplication, with better performances, whatever exactly you are after).