• The flip-flop operator
• The ideas of $\, $", and others.
• chomp vs. chop and where each is good
• Templates. They're not just for HTML! (Useful for reading as well as writing.)
• Data-driven parsing.
• Functional parsing (different from data-driven). tilly wrote something very cool on this topic regarding HTML-like parsing with functional programming.
• When to use a regex vs split vs unpack.
• How to use unpack! (I still don't get how to use it ...)
• The Burrito principle. (Very cool!)

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Could you expand on how split, pack, unpack and regexes are related? I feel there's something to what you say, but I can't at all pin it down.

split, unpack, and regexes are all ways to parse a given line of data. Each is useful in different circumstances. For example:
• split is more useful with delimited lines, such as tab-delimited or comma-delimited. (However, using a module like Text::CSV is better for delimited text. This is because of lines like "abcd,'Smith, John', blah" - the comma in the quotes is part of the item, not a delimiter.) Now, one could use a regex here, but the regex is harder to understand, and even harder to get right.

      my @items = split $delim, $line;
  #### vs. (and I know this will make mistakes
      my @items = $line =~ /^?([^$delim]*)(?:${delim}$)?/g;
  [download]

• unpack (if you understand how to use it!) is really good with data that is formatted, like so many columns is the first thing, so many the second, etc. This is often data from a mainframe.

  Again, you can use a regex here, but you have to roll it for it to be maintainable. (I'd put an unpack example here, if I was comfortable knowing how to work it.)

  my @columns = ( 20, 10, 25, 5, 2, 2, 20);
  my $regex = map { "(.{$_})" } @columns;
  $regex = qr/^${regex}$/;
  
  my @items = $line =~ /$regex/;
  [download]

For every example I give on different parsing needs, there is a module on CPAN that does it better, faster, and safer. I personally would never hand-parse data in production. Heck, you can use CGI to parse HTML pages without even having an http server!

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Here's some food for thought.

Check this article and the relative meditation on this site.

My personal advice is related to database programming.

You should explain when to use perl and when to delegate the task to a database engine. For example, comparing large lists of records and making statistics out of them is a task for a DBMS, rather than a candidate for a perl script. Of course, in such case you should introduce the beauty of the DBI.

Also the opposite cases are interesting, when people try to do in SQL things that should be better left to perl. As an example, parsing and cleaning data before storing it into a database is a perfect job for perl, which is the best companion tool for every database administrator.

 _  _ _  _  
(_|| | |(_|><
 _|
[download]

%months = (
   Jan => [1,  '31'], Feb => [2,  ''  ], Mar => [3,  '31'],
   Apr => [4,  '30'], May => [5,  '31'], Jun => [6,  '30'],
   Jul => [7,  '31'], Aug => [8,  '31'], Sep => [9,  '30'],
   Oct => [10, '31'], Nov => [11, '30'], Dec => [12, '31']
);

# Get number of days in Jan
$days = $months{Jan}->[1];
[download]

# in numerical order
for ( sort { $a->[0] <=> $b->[0] } keys %months ) {

# or reversed
for ( sort { $b->[0] <=> $a->[0] } keys %months ) {
[download]

Here is a much better data structure that also shows you how you can set a hash value to the return val of a sub and thus give feb a correct value. It also shows semi dynamic hash construction using map FWIW.

my @month_names = qw( undef, jan feb mar apr may jun jul aug sept oct 
+nov dec );
my @month_days = ( undef, 31, &feb_days, 31, 30, 31, 30, 31, 31, 30, 3
+1, 30, 31 );

my $months = {
    name   => { map{ $_, $month_names[$_]} 1..12 },
    number => { map{ $month_names[$_], $_ } 1..12 },
    days   => { map{ $_, $month_days[$_], $month_names[$_], $month_day
+s[$_] } 1..12 },
};

print "Month number 4 is ",   $months->{name}->{4}, "\n";
print "Month mar is number ", $months->{number}->{mar}, "\n";
print "Days in month feb ",   $months->{days}->{feb}, "\n";
print "Days in month 4 ",     $months->{days}->{4},   "\n";


use Data::Dumper;
print Dumper $months;

sub feb_days { check_leap_year( get_year() ) ? 29 : 28 }

sub get_year { (localtime())[5] + 1900 }

sub check_leap_year {
    my $year = shift;
    my $leap_year = 0;
    $leap_year = 1 if $year % 4 == 0;
    $leap_year = 0 if $year % 100 == 0 and $year % 400 != 0;
  return $leap_year;
}
[download]

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

and make a note that if you start the script in December of a leap year it will be wrong come February. but i guess you would use a module in that case.

One good thing you might want to mention is the usefulness of grammars. Modules like Parse::Recdescent are enormously useful for parsing highly complex data. You could even talk about them in contrast to "The Burrito Principle" (which is the greatest buzzword ever!), since grammars are usually written from a top-down approach. However, even with grammars, you could still apply the Burrito Principle: it is usually best to work on the most complicated/important subrule within a set of subrules first, as others in the set will usually follow easily.

Update: Cleared up my thoughts.

-derby

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.

Data mining also has a more general meaning of "sifting through a bunch of information (e.g. on the Internet) for useful data". The means of doing this are left up to the miner, although it usually requires the things you mentioned.

--
Allolex

I agree with allolex. My wife has done some amazing data-mining for job-boards when I was searching for a job. (Of course, this was all done before she learned how to cut'n'paste, but she's a smart one! *grins*)

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Obviously people use it that way, but it's still a bad idea to name your module that, for the simple reason that when someone goes searching CPAN for a data mining module, they won't want to find yours, because it doesn't do what they want it to. (s/module/whatever/ as appropriate.)

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.

Incidentally - are there any good references to getting into "Data Mining" in the sense that you used?

---

Seeking Green geeks in Minnesota