Creating a task scheduler from log file

KevinBr has asked for the wisdom of the Perl Monks concerning the following question:

I need to test a Unix development server by reproducting the load seen in production. The activity in production is written to a daily log file. My plan is to copy a production log file to the development server, read in the log file, and perform the same actions in development. My question is the best method to schedule the actions. The time stamp is included in the log file. I know how to parse the log and perform the actions, but I am not sure what to use to schedule the actions so they accour at the same time on the development server that they occured on the production server. I could use the Unix "at" command but that does not seem very efficient.

Comment on Creating a task scheduler from log file

Replies are listed 'Best First'.
Re: Creating a task scheduler from log file by CountZero (Bishop) on May 27, 2011 at 18:00 UTC
Forgive me my ignorance in things Unix, but why would the `at` command be not very efficient? CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply] [d/l]
Re^2: Creating a task scheduler from log file by KevinBr (Acolyte) on May 27, 2011 at 18:09 UTC
I expect to have about 100,000 events during the course of one day. I don't know if there are limitations to the number of at jobs I could schedule or the degree of difficulty removing them if I need to terminate the testing early.	[reply]
Re^3: Creating a task scheduler from log file by CountZero (Bishop) on May 27, 2011 at 18:22 UTC
or the degree of difficulty removing them if I need to terminate the testing early Removing scheduled jobs is as easy as calling `at -l` and after some perlish manipulation of its output feeding it to `at -r` CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply] [d/l] [select]
Re^3: Creating a task scheduler from log file by CountZero (Bishop) on May 27, 2011 at 18:17 UTC
I confess I do not know if there is a hard limit on the number of `at`-jobs, but that is easily circumvented, e.g. by splitting your huge list of `at`-jobs into separate batch-files and then having a master batch `at`-job file which calls (at appropriate times) the sub-`at`-job-batch files. CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply] [d/l] [select]