Homework Questions Wanted

Limbic~Region has asked for the wisdom of the Perl Monks concerning the following question:

All,
One of the many tasks of the team I lead at work is making a technical recommendation for changes to the production environment. One of the biggest issues is with "scripts" that are to be used for data gathering running in cron jobs or sysadmin tools. The issue is that these are not part of the production application and haven't necessarily gone through the same rigor. In fact, they aren't even being written by developers in most cases.

To that end, we have decided to offer peer training with a focus on scripts (perl, shell, batch, etc) with the possibility of branching out into other areas later such as improving SQL skills. This is completely voluntary with the direction the classes take based on input from the participants. The first session was a huge success but with with such large varying degree of existing knowledge, we want to be sure to have something to offer everyone. A number of participants have said they don't want to slow down the class and are willing to pick up the basics on their own.

To that end, we decided to offer "homework" assignments which are completely outside the class. These should be simple enough that the beginner's can accomplish them but with enough "extra credit" that the intermediate folks can still find a challenge too. Here is our first example:

Scenario:

You are paying bills and staring at your phone bill trying to figure out why it is so high when the phone rings. It is a telemarketer telling you how much money you could save if switch to their service. You kindly tell the person that you are not interested and, like every good telemarketer, they ask you what is the one number you call the most because they will let you call that number for free. You are not to be dissuaded so you just hang up the phone. As you walk back to your desk still staring at the phone bill you wonder what the answer to the telemarketer’s question really is.

Problem:

Parse a text representation of your phone bill and be able to answer the telemarketer's question. You can view your phone bill online but when you save it as HTML, you realize your perl skills aren’t quite good enough to parse it. It can be saved as a PDF but that’s isn’t any help either. You could print it, scan it using OCR and then convert it from a Word document to a text document but you are lazy and that seems like too much work. You decide the simplest solution is to open a text editor and copy/paste the data.

Input Description:

The text file is tab delimited and looks like:

Number    Direction    When    Duration    Cost
(123) 456-7890    Inbound    Fri, 7/24/2009, 8:53AM    5 minutes    $1
+.47
(456) 789-0123    Outbound    Sun, 10/1/2009, 12:48PM    1 minute 34 s
+econds    $0.12
[download]

Extra Credit:

After discovering the number called most frequently, you realize you have never called that number. You look closer and all the calls happen when you are at work. You begin to suspect that perhaps your significant other has something to hide. You also realize that the phone bill is still higher than you can explain. What other gremlins are hiding. You decide to data mine this phone bill dry. Here are some questions you might want to answer:

Which numbers are being called while you are at work? (assume 8:00AM - 5:00PM M-F)
Is anyone calling and hanging up?
- Is there a way to determine it is just a prank caller?
- Is it an automated telemarketer hanging up when the answering machine comes on?
- Is it someone who only hangs up when you answer the phone but other times talks for longer?
For any given number X, what is the rate to call that number?
- Does time of day affect the cost?
- Is the charge per minute or per second?
- If not per second, how does the phone company round (any partial round up for instance)?
Are you being over charged for any calls?
- Perhaps your plan charges a fixed rate and you discover a call that exceeds it (optional input)
- Perhaps all calls fall into the same rate category but a few were charged more
Which number is the most/least
- Called most frequent (same as telemarketer's question)
- Called with longest duration (individual and aggregate)
- Costs the most (per minute as well as total overall cost)
- Received calls with the most frequency
- Received calls with the longest duration (individual call and aggregate total)
- Same questions as above except s/most/least/g and s/longest/shortest/g
What if you answered the questions above but wanted the top or bottom N calls instead of the extreme?
How much time is spent on the phone (while you are home vs at work)?

Thos are not the only bumps along the road. You should also make this code as robust and flexible as possible. Have you considered

What international numbers will look like?
That not all rows will have data you expect (header/footer for instance)
That you may need to be able to specify command line parameters instead of re-coding every time you want to ask a different question
Does your duration conversion routine handle all possible inputs?
Have you considered what would happen if a call lasted 0 seconds?

We didn't spend a lot of time coming up with this problem but it seems to be a good fit for the beginner and the intermediate. We are hoping you might have some other good ideas?????????

Cheers - L~R

Comment on Homework Questions Wanted Select or Download Code

Replies are listed 'Best First'.
Re: Homework Questions Wanted by planetscape (Chancellor) on Aug 04, 2009 at 21:53 UTC
Please see Re: Real Life Perl Exercises, where I have compiled a list of homework etc. questions. HTH, planetscape	[reply]
Re: Homework Questions Wanted by halfcountplus (Hermit) on Aug 04, 2009 at 16:10 UTC
"Intermediate Perl" (Schwartz et. al.) has a great series of similar exercises in it, with solutions. It is explicitly object-oriented, but also contains the best explanation of essential things like grep and map I've ever seen. The goal is module writing, which looking at the exercise above I would say it might be best done that way, anyway. And it's not too thick. If your participants are interested in doing perl homework they'll love it.	[reply]
Re^2: Homework Questions Wanted by Limbic~Region (Chancellor) on Aug 04, 2009 at 17:44 UTC
halfcountplus, ...which looking at the exercise above I would say it might be best done that way First, thank you for your response. We have provided a long list of perl resources (most free) to include book recommendations. I do want to reiterate that the point of these "homework" assignments is to give achievable tasks for beginners as the primary objective and "extra credit" for intermediate students as secondary. Writing modules (OO or otherwise) is beyond the scope of what I consider beginner perl. I do own a copy of Intermediate Perl (back when it was hideously called "Learning Perl Objects, References & Modules") but haven't found the exercises a good fit for the beginners. Cheers - L~R	[reply]
Re: Homework Questions Wanted by Bloodnok (Vicar) on Aug 04, 2009 at 16:07 UTC
Just a thought, but it appears to me that we have regular questions regarding running scripts etc. requiring non-standard &/or login environments, under cron(1M) - so it would seem to me to be too good an opportunity to pass up un-exercised e.g. Ignoring the splendid Log::Log4perl library, write a module to implement log management related functionality - typically rotation Install it into a non-standard perl library Write a script to provide a CLI for it [the module] Write and test a cron entry to utilise the newly written script A user level that continues to overstate my experience :-))	[reply]
Re: Homework Questions Wanted by raisputin (Scribe) on Aug 04, 2009 at 18:55 UTC
Not sure that I can offer anything useful here, but as pretty much a beginner myself, these are some things that I have needed to do in our business to better enable our after hours technical support since they cannot have access to some of the systems that hold information, or it was to time intensive for them (they are lvl 1) to do on a regular basis via our RT system. Maybe you can come up with some ideas from those things I have implemented? Provide an easy way to watch dial-up logins via a web page Connect to another system and retrieve some arbitrary information delimited by lines, parse that information and display it in an easy to read table format highlighting anything that is out of the ordinary. As an example: Our cable modem system reports power levels for upstream and downstream. There are certain values that are within spec. Our techs needed a simple way to know that they would be unable to solve the problem if one of those values was out of spec, so in the case that they are out of spec, we highlight that value so they can escalate the call to Lvl 2 technical support. In order to get these levels, we must create an ssh connection to the CMTS, query the modem by MAC address, then parse the values from the resulting dataset and display them to the tech so they are obvious. On our mailservers (that need to be upgraded LOL) sometimes the spamassassin process just quits (No, I am not in charge of these, thanks). So I wrote a script that monitors each server to ensure that process is running. If it is not, it will send an email to the mail admin to let him know. On our mamilservers again: Oftentimes users call in and say that they did not get an email from joeblow@somedomain.com. Almost without fail, this means that the spam filter has caught it and sent it to their junkmail account (which users never check, go figure). Searching this was difficult because more often than not users didn't know exactly when the mail was sent saying things like "last week sometime". To facilitate a time savings, I wrote a perl script that executes a shell script on each server. The perl script takes 2 arguments date as yyyymmdd and emailaddress or messageID. Users can enter a date as yyyy, yyyymm,yyyym, yyyymmd or yyyymmdd so the command syntax is: `msearch 20090803 email@domain` searching for mail from someone that was not received or `msearch 20090803 1MY1zZ-0004fr-46` searching for a particular message after the result of the address search. I still need to update this to allow selection of a given message on a given mailserver, and add an argument to restrict the search to a given set of servers instead of all of them, but there has been no time to do so thus far In our cable modem provisioning, we used to do it by hand and provision a specific modem for a specific customer before it went out the door. I wrote a small web-based application and with some configuration changes in our server, we no longer have to provision them before they go out. Our techs now go to a customers home with a modem that has been generically provisioned as a "New Install". When they connect the modem at the customers house, they are forced to a website in which they fill out some basic information: Customers Name, Telephone Number, email address and what type of service they have and whether this is actually a New install or if it is a customer whose modem we are swapping out for some reason. On submit, the web app, through some perl trickery writes out a provisioning file in /tmp then uses some provided CLI tools to connect to the provisioning server, and make the required changes to the modem. It then sends an email out to our admin and customer service teams to ensure that it was properly provisioned. Sounds like a long process, but the whole thing takes less than 2 minutes on average and saves us hundreds of hours Password Generator :) Create a simple Password generator to create 8 character passwords. because l and 1 and O and 0 often get confused when looking at them, ensure that if there is an "l" (el) in the password that it is always uppercase and if there is an "O" that it is always lowercase so that they are not confused with 1 (one) and 0 (zero) I am sure I can think of more things that I have done, just not right now. Being a beginner, some of these were still fairly easy while some have been challenging in the extreme, such as provisioning cable modems via a web page as that required me to learn not only how to write files, but how to write them properly for the CLI tools provided by a particular software vendor. Another challenging part of the provisioning was that if it is a modem swap we pull all of the information from the provisioning server based on MAC address and pre-fill in the form for the tech. This was challenging because it required me to learn how to use the provisioning system effectively from the provided cli tools instead of the GUI. It was fun though I have to say. I have now been programming perl for about 1.5 years and am getting more and more comfortable with it. I am sure that some of my code, the old pro's would look at and laugh until they puked, but at the same time, that is one of the things that I find beautiful about perl, I can write the code in such a way that I have a complete understanding of it at this point in my learning, and as I get better and learn more, I can go back and change it to be smaller, better and more "correct". I don't know who your students will be or what their programming ability is, but for those with little to no background in programming, I would offer this: Perl is an enabling language. Because you can write things in many different ways, your code and mine may not be even remotely similar in approach or execution. As time goes on you learn more and your code improves. Each time you write something it gets better, and if you are stuck: perlmonks!!!!	[reply] [d/l] [select]
Re: Homework Questions Wanted by Your Mother (Archbishop) on Aug 04, 2009 at 16:44 UTC
If shell/sysadmin stuff is the focus, this might be a better choice than the O'RLY books (or at least a choice to augment lessons and a good book to have sitting around the office): Minimal Perl; it is *nix-centric.	[reply]
Re^2: Homework Questions Wanted by Limbic~Region (Chancellor) on Aug 04, 2009 at 18:02 UTC
Your Mother, Perl for System Administration is already on our list of resources and we will add Minimal Perl - thank you. We don't want to restrict the "homework" to just sysadmin tasks as these are things that truly are intended to be worked from home. While it most be the most applicable and tangible, not everyone has an AIX server at home to play with - besides, variety is the spice of life. Cheers - L~R	[reply]
Re: Homework Questions Wanted by biohisham (Priest) on Aug 04, 2009 at 16:10 UTC
I have been thinking about the same issue too, a lot, but from a learner's perspective, since I am finding it daunting to come with examples and problems that match my progress rate, I am reading off this huge technical book and it scarcely has any examples but it is rewarding, I use my imagination to come with programming scenarios and start acting accordingly, of course, sometimes I become partial and skip some of the areas I need to focus on, which is not the best thing to do, so this is like a great initiative. Best of Luck Excellence is an Endeavor of Persistence. Chance Favors a Prepared Mind	[reply] [d/l]
Re: Homework Questions Wanted by raisputin (Scribe) on Aug 04, 2009 at 16:32 UTC
TOTALLY OFF TOPIC REPLY ROFLMAO!!! HOLY CRAP! "...After discovering the number called most frequently, you realize you have never called that number. You look closer and all the calls happen when you are at work. You begin to suspect that perhaps your significant other has something to hide..." Did you take this scenario from a call of mine from like 7 or 8 years ago? That was EXACTLY what happened with my ex-wfie LOL!!!!!! Of course she denied it, but a litte keystroke recorder on her computer and a weeks worth of collecting the resulting email logs confirmed my suspicion. Good riddance to her! LOL LOL LOL	[reply]
Re^2: Homework Questions Wanted by Limbic~Region (Chancellor) on Aug 04, 2009 at 17:52 UTC
raisputin, I am glad you found our story line humorous as that was the intention. We hope it is effective. No, this wasn't taken from your or anyone else's real world experience. We did think it very plausible and that was what we were going for. We are encouraging participants to bring their real work related issues as the topic for a lesson at work. Working on something real often reinforces a lesson much more than some contrived example from a book. Unfortunately, this won't work for our "homework" because of the sensitive nature of what we work on. We still wanted to provide examples that were realistic, practical and left some questions intentionally unasked to encourage people to go out on their own. Cheers - L~R	[reply]
Re^3: Homework Questions Wanted by Narveson (Chaplain) on Aug 04, 2009 at 21:46 UTC
When Germany liberalized its telecoms market in the nineties, I worked for one of the American firms that jumped in to compete not just on price, but by offering businesses previously unknown services like itemized bills. I enhanced the bill with a "Most Frequently Called Numbers" report, and consternation ensued.	[reply]
Re: Homework Questions Wanted by WBF3 (Initiate) on Aug 05, 2009 at 19:31 UTC
I have noted that another very good homework assignment is to give the student a working block of code and then ask what it does. This works especialy well when I am trying to learn a language Does anyone have any suggestions on which code found here that I should try to figure out?	[reply]