ravi45722 has asked for the wisdom of the Perl Monks concerning the following question:

-rw-r--r--. 1 root root 2.1G Feb 21 16:00 SMSCDR_POSTPAID_160221150000 +_10.102.39.37_AS.log -rw-r--r--. 1 root root 1.5G Feb 21 17:00 SMSCDR_POSTPAID_160221160000 +_10.102.39.37_AS.log -rw-r--r--. 1 root root 2.3G Feb 21 18:00 SMSCDR_POSTPAID_160221170000 +_10.102.39.37_AS.log -rw-r--r--. 1 root root 1.7G Feb 21 19:00 SMSCDR_POSTPAID_160221180000 +_10.102.39.37_AS.log

I have written a program to process these files for some counters. That program is OK. But now I want to know which line is processing now in the file. For that I cant print it on terminal because it is running in cronjob. and I cant maintain log files for just line number. What are the best practice for these????

Replies are listed 'Best First'.
Re: Want to know Line number of the input file
by Discipulus (Canon) on Feb 24, 2016 at 12:03 UTC
    Hello,

    line number of the current file handle is stored in the special variable $.

    You may note that $. is reset only by an explicit close of the filehandle, so if the same filehandle is reopened $. continue grows between files. This can be avoided with close ARGV if eof that is very useful for oneliners.

    perl -lne "print qq($.)" two_lines.log 13_lines.log 1 2 3 #<--here starts second file 4 5 6 7 8 9 10 11 12 13 14 15 perl -lne "print qq($.); close ARGV if eof" two_lines.log 13_lines.log 1 2 1 #<--here starts second file 2 3 4 5 6 7 8 9 10 11 12 13

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
Re: Want to know Line number of the input file
by marto (Cardinal) on Feb 24, 2016 at 12:08 UTC

    $. perlvar:

    test.txt:

    line 1 line 2 line 3 line 4 line 5 line 99

    ln.pl:

    #!/usr/bin/perl use strict; use warnings; open(my $fh, "<", "test.txt") or die "cannot open < test.txt: $!"; while (my $row = <$fh>) { chomp $row; # do something print "line number $.\n"; }

    output:

    line number 1 line number 2 line number 3 line number 4 line number 5 line number 6

    update: forgot to add the output.

Re: Want to know Line number of the input file
by FreeBeerReekingMonk (Deacon) on Feb 29, 2016 at 23:04 UTC
    Hello Ravi,

    what I would do is use signals to give a nudge to the long running process...

    I have included a print statement (you can make that to log one single entry in a logfile) OR, also fancy, modify the name of the script $0 directly, hence, you get the information through ps -ef (used by some programs). There are more ways, like named pipes, etc.

    #!/usr/bin/perl my $SHOUT = 0; $SIG{USR1} = sub { ++$SHOUT; }; my $PROGRAM = $0; print "$PROGRAM started with pid $$\n"; my $counter = 0; while(1) { $counter++; if($SHOUT){ print "$0 says: $counter \n"; $0 = $PROGRAM . " counter=$counter"; $SHOUT = 0; } sleep 1; }

    The nudge is given with a kill -USR1 $PID

    Update:

    The normal way is to have a second program that asks for ETA and status. This second program send the signal, then both programs then initiate a temporal TCP connection to send/receive the information. But from what I read in your post, you want to keep it simple. The other way is to only update your linenumber logfile each 10 seconds. you will need $NEXTUPDATE = time()+1000*10 and do compares if(time()>$NEXTUPDATE) This will use much less IO than logging each line.

    edit2: added missing my's. And Do'h, suggesting same things as peers... sorry for that.

    *important note*: Do not use IO in signals. Your script could coredump if you open files and such. Do the work outside.

Re: Want to know Line number of the input file
by RonW (Parson) on Feb 29, 2016 at 21:58 UTC

    You could maintain a status file. Open it once, at the beginning, then seek($stat_hf, 0, 0); just before you write the current status to the file. This is a lot of extra IO, so you might want to do it once every 5 or 10 input lines.

    Unfortunately, some operating systems might auto-lock the status file against even reads from other processes. The preferred "fix" for that is to open it with options to only lock the file for writes. (but I don't know how to do that.) A common work-around is to close and re-open the file for each update. This incurs much more extra IO, so you'd only want to update once for every 50 or 100 input lines.