jtrousde has asked for the wisdom of the Perl Monks concerning the following question:

Hey guys - sorry for the long title. Long story short, I'm new to Perl (so new, in fact, that I have yet to write a single line). But it's good to learn, right?

This is the initial problem I've been presented with:

I need to locate the maximum value in the 2nd column of a file. I then need to subtract off a value at an initial time (given in the first column). The initial time is 3000 seconds, but in no file is the value "3000" ever presented. Rather, we take the first value which transitions from 2999 to 3000. This needs to be done for quite a number of files in a directory.

I then need to export these differences to a document.

Here's an example of what I'd like to do with some data:

2999.98 280 2999.99 281 3000.01 280 3000.02 290 3000.03 300 3000.04 310 ... 4250.01 300 4250.02 290

The program would take 310 from column 2, then go to column one and find the value "3000.01", take the value of 280 in that row, and subtract it from 310. It would then write the difference (30) to another file. After it's finished with every file in a directory, it would save that new file to a separate directory.

Now, I've got plenty of experience in other languages to be able to catch on quickly (and honestly, I'd like to do it in another language, but my heads are tied on my software of choice).

Let me know, guys!

As a side note, I have the "Learning Perl" book from Schwartz, Phoenix, and Foy. If anyone is extremely familiar with this book and offers a response, I'd like to know which sections might be particularly relevant to the coding solution.

Thanks everyone!

  • Comment on New to Perl - locating max, subtracting initial, printing output, all across several files in differing directories
  • Download Code

Replies are listed 'Best First'.
Re: New to Perl - locating max, subtracting initial, printing output, all across several files in differing directories
by choroba (Cardinal) on Feb 04, 2015 at 14:50 UTC
    Hi jtrousde, welcome to the Monastery!

    You can study Perl from the books, but its documentation is also a good source of knowledge. Start with perlintro, of course. For the task, you should also read split, open, glob or opendir, List::Util (for max), and print.

    If you get stuck, ask here!

    لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
Re: New to Perl - locating max, subtracting initial, printing output, all across several files in differing directories
by blindluke (Hermit) on Feb 04, 2015 at 17:49 UTC

    Welcome to the Monastery.

    "Learning Perl" is a great book to read cover to cover, doing the tasks, reading the chapters in order. Not the best reference to pick out sections from and taking out stuff specific to the task you have at hand. There are better sources of knowledge for this type of search, a great point to start would be the Perl FAQ. Read the FAQ, try to get the bits you need, play around, break stuff, and read "Learning Perl" at your own pace. Make sure that the edition you have is at least the fourth one, and if you have the most recent one (6th edition), take everything you read in Chapter 15 (the one about the smart match operator) with a grain of salt. Some things have changed.

    As to your problem, here is a small example that could get you started.

    #!/usr/bin/perl use warnings; use strict; my $maxvalue; my $value_at_3k; while (my $line = <DATA>) { my ($time, $value) = split ' ', $line; if ($value > $maxvalue) { $maxvalue = $value; } unless (defined $value_at_3k) { if ($time > 3000) { $value_at_3k = $value; } } } print "Maxvalue was $maxvalue \n"; print "Value at 3k was $value_at_3k \n"; __DATA__ 2999.98 280 2999.99 281 3000.01 285 3000.02 290 3000.03 320 3000.04 350 3000.05 325 3000.06 275 3000.07 270 3000.08 260 3000.09 235

    Start by reading the FAQ, then try to change the code so it reads the data from a file (with the filename declared explicitly in the code). Then wrap the code in a subroutine that accepts the filename as a parameter. Then try iterating over a directory - first printing the filenames, then passing them as arguments to your sub. Then wrap it up. Good luck with the task ahead of you.

    And it's good to learn. You are very right about this.

    - Luke

      Thanks for the responses! I can sort of see what's going on with your sample code. I assume I'm doing to replace DATA with the directory. Maybe not. I'm trying to make sense of the FAQ at the moment.

      I suppose I actually need to modify my original intentions.

      The data I need to read exists in directories within a parent directory. For example, I have directories 1, 2, 3, ... , 10, all within a parent directory (call it /users/parent/).

      Within those folders is a file (file will always have the same name) with six columns with headers. I'm assuming the $value and $time are the headers of columns within my data file.

      Additionally, I no longer need the difference. What I'd like to do is just pull the maximum value of a specific column within a specific range of column 1 (3000 to 3640), as well as the value in column two corresponding to the time closest to 3000 in column 1. These will be exported as two different columns to another file: max value in column one, "initial" value in column 2.

      Ugh. Perl's notation is so different from what I've seen elsewhere. I feel like I'm drinking from a firehouse for a fairly simple problem.

      EDIT: Working on it, btw. But it will take some time. Thanks for getting me started :)

Re: New to Perl - locating max, subtracting initial, printing output, all across several files in differing directories
by Laurent_R (Canon) on Feb 04, 2015 at 19:52 UTC
    Cross-posted on the Perl Gurus forum (http://perlguru.com/gforum.cgi?post=80875;#80875) and apparently also on Stackoverflow, but I do not know where.

    Please be aware that there is nothing wrong with cross posting per se, but it is considered polite to inform people about that, to avoid duplication of work at various places of the Internet. If your question was solved somewhere else, it would be stupid for me to waste time trying to give you an answer here.

    Je suis Charlie.