Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^2: Time modules

by mikejones (Scribe)
on Feb 24, 2009 at 18:44 UTC ( [id://746072]=note: print w/replies, xml ) Need Help??


in reply to Re: Time modules
in thread Time modules

I think Time::Progress is what I need or something similar. I am playing with it now. To answer your questions, IDK for sure.

I am using File::Find on some small and some very large filesystems, 30Gb for example. I could use the Unix time binary to give me total time elapsed.

The units would be the number of files to look at for a specified size in any one filesystem

I could adjust the expected time, something solid is ideal but "something" is desired, maybe a rough estimate???

The interface for times lapsed and time to completed is OK to look like

Elapsed Time: 0:20 min:sec, ########..........................20.0%
nothing fancy. Does min and max represent seconds here ? $timer->attr( min => 0, max => 100 ).

Is the for loop here used just for the printing of #s?

CPAN does not say:

min This is the min value of the items that will follow (used to calcu +late estimated finish time) max This is the max value of all items in the even (also used to calcu +late estimated finish time)

use strict ; use warnings ; use File::Find ; use Logfile::Rotate ; use Time::Progress ; $| = 1 ; ###-- autoflush to get \r working --### my $timer = new Time::Progress ; $timer->attr( min => 0, max => 100 ); $timer->restart; print "\n"; for my $incr ( 0 .. 20 ) { ###-- print progress bar and % done --## +# print $timer->report( "Elapsed Time: %L min:sec, %40b %p\r", $incr + ) ; sleep 1 ; } ### MAIN CODE HERE ### <snip> logroll ; ### END OF MAIN CODE ### ###-- stop timer --### $timer->stop ; ###-- report times --### print $timer->elapsed_str ;

thank you!

Replies are listed 'Best First'.
Re^3: Time modules
by ELISHEVA (Prior) on Feb 24, 2009 at 19:25 UTC
    The documentation is very confusing, but between reading the source code and rereading the docs, here's how it works:
    • min is the starting unit, e.g. 0
    • max is the ending unit, e.g. 100 or 1000
    • or in your case, maybe number of files or total number of Gb
    • report($format, $cur) - prints out how much has been done so far using the format $format. $cur is the number of units consumed so far, i.e. the number of files or number of Gb examined so far
    • time estimates are calculated by comparing the lapsed time to $cur/($max - $min) and scaling up appropriately.

    So the usage would be something like:

    1. create a progress object with appropriate min/max
    2. in a loop,
      1. do N units of work
      2. after each N units have been completed, call report($format, $cur) to report how many units have been completed so far

    BTW - I also noted the docs show report like this report(format, [current item]). I'm not quite sure what was intended by that. According to the source code, the call should look like this: report($format, $currentItem)

    Best, beth

      Beth is on it!

      I would add that be careful as any kind of user display is a VERY expensive operation - more expensive than even opening a file.

      I usually put a thin layer that counts N units of work. This is called after every work unit. When N units are reached (or time window, etc), this routine calls the UI. The main point here is that the UI is SLOW. If say you are doing 10,000 things and have a progress bar with 0-100%, there will be a HUGE performance difference between just calling that UI 100 times vs 10,000 times! If your program makes progress very quickly, do a test case just to see how fast the progress can go from 0% to 100% without doing any "real" work. You will be amazed. In this type of case, the CPU power required to calculate the time since last display will be dwarfed by the CPU power required to actually display something. This is true on command line as well as GUI. Some folks think that command line display is fast, but that is not true. In some cases this can be even slower than a GUI display.

      I hate this Microsoft file copy display. It it just a picture and is a separate process unrelated to the actual work being done. If you have a choice, always display something that is related to the amount of work that has been done, not just a "I am working" message.

      I made some progress, however I was not sure what you meant in number 2. Since I am using file find, my units would be every file greater than 25Mb for example.
      <snip> else { my ( $size_input ) = pop @_ ; if ( $^O !~ /(?:)dec\w+/i ) { my ( $dev,$ino,$mode,$nlink,$uid,$gid ) ; (( $dev,$ino,$mode,$nlink,$uid,$gid ) = lstat($_) ) && ( $dev >= 0 ) && !( $File::Find::prune |= ($dev != $File::Find::topdev ) ) +&& ( int(((-s _) + 1023) / 1024 ) >= $size_input ) && push ((@large_files), $name) ; $number_of_files = scalar @large_files ; $timer->restart( min => 0, max => $number_of_files ) ; print $timer->report( "%45b %p\r", 10 ) ; }
      ### The output is not entirely accurate. For a large filesystem this is painted across my screen and counts down. For smaller filesystems, numbers are not calculated quickly enough and all I see is n/a. For mid-size filesystems all I see is 100% ###
      ##################################........... 76.9% #########################.................... 55.6% "output of the file find hereafter"
      In the code  print $timer->report( "%45b %p\r", 10 ) ; does 10 represent % points to increment or decrement in his calculations? thank you
        Neither. It means number of points completed so far. So assuming you are updating your progress bar after each 10 files and you had 50 files total, you'd set min=0, max=50 and each successive call would be something like this:
        print $timer->report( "%45b %p\r", 10 ); #10 out of 50 files print $timer->report( "%45b %p\r", 20 ); #20 out of 50 files print $timer->report( "%45b %p\r", 30 ); #30 out of 50 files

        Best, beth

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://746072]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (5)
As of 2024-03-29 07:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found