But yes it's possible, you basically need a constantly running server/demon which does your task.

-Pro: no Perl startup time, no script load time, no compile time overhead

-Con: Socket/Port communication, additional effort needed to make your script stateless to avoid side effects

For example, you can have a look at Perl::Tidy which can be run in server mode.

> (it has 15000 pages to build, across 6 template sets ... so takes quite a while to run)

I can't see the benefit here, except if you really try to start Perl for each single page ...

Thanks for the reply. OK, so sounds like it wouldn't work. I just wanted to see if it was possible before I start digging into the guts of the modules to speed things up (I just found a load of redundant code in a module that we don't use any more, so that wouldn't have helped). Maybe I need to look at some mySQL indexes as well to see if I can speed anything up there. Thanks anyway :)

> OK, so sounds like it wouldn't work.

I can't tell because you need to be more specific.

WHAT do you think could be optimized away?

Do you REALLY restart Perl for each single page?

Cheers Rolf
_{(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery FootballPerl is like chess, only without the dice}

I looked at Rolf's reply and I agree with it.

In addition, I'm not sure what you mean by "Maybe I need to look at some mySQL indexes as well to see if I can speed anything up there.". The first thing you should look at is decreasing the number of transactions per second if at all possible. 250K updates or inserts as a single transaction is just fine. Truly enormous performance increases can potentially be obtained. The subject of SQL indices is complex and quite a bit of experimentation and benchmarking may be required to achieve what you want. You should be aware that "over indexing" your DB can slow things down by confusing the query engine! I have seen cases where taking an index out increased performance significantly.

Oh, as another thought, many things can come into play with DB performance. Might want to look at How MySQL Uses Memory.

Update:I looked again at the OP. "I'm trying to speed up a script that runs on a cron." I don't see the exact connection between that cron job and whatever is serving up your webpages or why the cron job needs to be faster?. It almost sounds like you update 16K web pages in a batch process via a cron job. These static pages are then served up as requested.

I have one chron process that calculates new static web pages once per hour, on the hour on a machine dedicated to the DB and calculations. The pages then get uploaded to the web server. Each page shows a "last updated timestamp". The amount of work to be done each hour varies - sometimes just a second or two, sometimes 15 seconds. For the user, if they see results from the last hour, they have to come back or hit refresh until this hour's results show up. In practice the users know that they should wait until the hour +1 minute to get the latest results.

At the end of the day, We (I am using the collective we) don't really understand your application and providing specific advice is problematic.

You should profile your script(s) to see where it's spending its time and where you should focus on optimizing.
https://metacpan.org/pod/Devel::NYTProf