Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Log parsing exercises for Perl beginner - reasonable?

by mikeraz (Friar)
on Apr 21, 2011 at 21:42 UTC ( [id://900725]=perlquestion: print w/replies, xml ) Need Help??

mikeraz has asked for the wisdom of the Perl Monks concerning the following question:

A co-worker is learning Perl through self study. He's borrowed my Llama book for a start. As part of the learning I've agreed to provide him with "real world" exercises.

Do the following exercises for analysing *nix log files seem reasonable to you? He seems to be having trouble with them. He does know about split and join and the basics of array and hash use. A few hundred lines of messages was snipped and dropped into the directory he's using for the exercises.

Problem #1: Read the log file messages. Print a list of: Count TimeStamp I'm looking for the number of log messages per hour of the day. S +omething like
144 Apr 10 04:00 280 Apr 10 05:00 Problem #2 Read the log file messages instead of counting the number of messag +es per hour, count the number of instances for each message. Something like 2 syslogd 1.4.1: restart (remote reception). 189 ntpd[3216]: synchronized to 192.168.1.15, stratum 3 1880 last message repeated 9 times Problem #3 Modify solution to problem #2 so that "last message repeated X time +s" adds to the count of the last message Problem #4 Modify solution to problem #3 to ignore the PID Meaning: ntpd[3216]: synchronized to 192.168.1.15, stratum 3 will report as ntpd: synchronized to 192.168.1.15, stratum 3 Problem #5 Modify solution to problem #4 to report on activity per hour Apr 10 04:00 - XXXXX xxxx ntpd: synchronized to 192.168.1.15, stratum 3 xxxx syslogd 1.4.1: restart (remote reception). replacing xxxx with the acutal numbers

Be Appropriate && Follow Your Curiosity

Replies are listed 'Best First'.
Re: Log parsing exercises for Perl beginner - reasonable?
by raybies (Chaplain) on Apr 22, 2011 at 04:02 UTC

    Did you help him decode the timestamps at all? That might hang him up. Has he any experience in programming at all?

    If not, then something as simple as looping or opening a file could be a novel concept.

    Hashes are a great thing to teach, but so are Perlvars in general, as they work quite differently than other programming languages.

    --Ray

      Your coworker may not be aware of how powerful regexes are, and may be trying to do a lot of the scan/analysis the hard way. Regex memory is a great tool.

      Does he know about substr?

      And if he's going to be doing much with logs, then an intro to Time::Local would be in order (once he gets his feet under him). Ask him to come up with the average time between log entries w/o writing a time arithmetic routine of his own - instead learn the beauty of timelocal().

      But to answer your question: no, these appear to be a reasonable start to learning Perl.

      He has some experience with coding and this is "yet another" attempt to learn Perl. He has demonstrated the ability to loop and open files.

      The timestamps, eg "Apr 14 10:18:22", should present no problem.

      The last time I peeked at his code he was replacing all colons with spaces so he could split on white space and then join the first three fields in the split array.


      Be Appropriate && Follow Your Curiosity
Re: Log parsing exercises for Perl beginner - reasonable?
by SitrucHtims (Novice) on Apr 22, 2011 at 01:44 UTC

    I don't think these are unreasonable. I am about 4 weeks old working with Perl myself, and I could accomplish these problems.

Re: Log parsing exercises for Perl beginner - reasonable?
by Voronich (Hermit) on Apr 22, 2011 at 14:07 UTC
    +1 for "Yep, these are just fine." And frankly they're pretty refreshingly real world compared to a lot of the overly academic nonsense out there.
    Me

      One goal for the exercises is to provide "real world" problems. I often see co-workers struggling with a question that could be answered with a bit of Perl. These same people have made small attempts to learn Perl in the past but they've never stuck with Perl due to a lack of identifiable real world application. I'm trying to get this guy to the point where he'll have the tool in his box and recognise where it's useful.


      Be Appropriate && Follow Your Curiosity
Re: Log parsing exercises for Perl beginner - reasonable?
by InfiniteSilence (Curate) on Apr 22, 2011 at 19:24 UTC

    Here are some links for the various popular learning theories. Also, if you Google 'learning theory' you will get more sites, but they will mainly fall into the same categories.

    The problem people have when teaching a computer language or technical subject is the (false) assumption that all people learn the same way. It simply isn't true. Some people require a more active approach; others are adult-type learners that want actual problems and immediate gratification; many Perl coders are self-learners and you only have to show them the manual/pod, etc. From your description you are using more of an adult-learner approach and you are getting feedback that it is not working. My recommendation is that you stop doing that immediately. Walk over and take the materials away and tell your co-worker that you are going to help him/her using a different approach. Then research the different approaches and choose one that better matches your co-worker's learning style.

    What happens if you don't do this? Your co-workers will likely hate this language. They will probably also start to dislike you for foisting it upon them.

    Lastly, I completely disagree with the use of log files for teaching elementary Perl. Simple data constructs, variable assignment, string manipulation, simple regexes, debugging and getting Perl installled/running are topics of primary importance because they must be used all the time. Parsing large files is not something you will do all of the time, and therefore should not be promoted too early any more than the use of sockets.

    Celebrate Intellectual Diversity

      Lastly, I completely disagree with the use of log files for teaching elementary Perl. Simple data constructs, variable assignment, string manipulation, simple regexes, debugging and getting Perl installled/running are topics of primary importance because they must be used all the time. Parsing large files is not something you will do all of the time, and therefore should not be promoted too early any more than the use of sockets

      Depends entirely upon the audience and lesson preparation, and what you're trying to learn, to wit, Seven Levels of Perl Mastery, Ten Perl Myths, Modern Perl Released as Free ePub

      Pick your path to mastery, the long road, Seven Levels; the short, Modern Perl.

      Sorry, gotta disagree on the idea that one doesn't parse large files often with Perl. At least in my experience it's been a huge part of what I do... I guess it really depends on the sort of jobs you do with Perl...

      I see your point though... In many respects, it's not the file that really matters, it's the approach of setting up a datastream and then looping over it, whether it be a file, or socket, or database queries...

      I think the exercise is fine, but when if he doesn't get it, I agree you've gotta back off and try a more basic approach. The nice thing about an exercise like this one is that it often teaches a lot of concepts at once, and can be extended... by simply tweaking it to add a new feature like Time::Local, or regex, or uniquifying hashes, etc...

      --Ray

      Thank you for the link to the learning styles information. That will be great help going forward. The guy I'm working with specifically requested exercises that would apply to the work we do. So the style of teaching and the subject matter both fit.


      Be Appropriate && Follow Your Curiosity
Re: Log parsing exercises for Perl beginner - reasonable?
by anonymized user 468275 (Curate) on Apr 22, 2011 at 14:07 UTC
    (Update: meanwhile it occurred to me that the stated skillset of the co-worker doesn't include regexps, so "hint see perlretut" might also help)

    The idea of "instances of each message" is a bit vague. For example, it doesn't say whether the IP as well as the date-time of the message may vary while still being considered instances of the same message. I'd specify exactly what is being allowed to vary within the implied relationship "instances of", even if it turns out to be just the time of the message.

    One world, one people

      So you don't think the time stamp free examples clarifies that point?

      Thank you for the observation. I'll be watching to see if he goes down the wrong path here.


      Be Appropriate && Follow Your Curiosity
        If you're saying that examples in one exercise should create a rule to be applied to another exercise, then an indication of that would help. OK I suppose I am more picky about expressing things thoroughly in documentation and educational materials than most people. On the other hand there is that rule about assumptions: ASSUME = ASS of U + ME

        One world, one people

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://900725]
Approved by Perlbotics
Front-paged by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (8)
As of 2024-03-28 19:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found