You might be unaware that WWW::Mechanize keeps a history of visited pages. If your script never quits and only ever keeps one WWW::Mechanize object around, that will accumulate a vast history over time, which will also consume more and more memory.

Of course, without seeing code, it's hard to tell.

This is the likely culprit. Either set stack_depth() on the Mech object, or periodically destroy the object and create a new one.

Thank you , this was indeed the problem

+1 on that one. Mechanize is great, but a hog.

I would consider "reference counting" one of the possible algorithms of garbage collection. Some others being mark-and-sweep and copying and their generational extensions. To me "garbage collection" means I do not have to deallocate memory explicitly.

I would actually like best a GC that'd count references, release immediately everything it can and run a mark&sweep or copy the still live stuff from time to time to clear up the forgotten self-referencing data. Especially since we are used to reference counting in Perl and mostly do make sure we do not forget to break cycles, but we tend to expect that objects are destroyed as soon as they stop being referenced.

Jenda


Support Denmark! Defend the free world!

I would call reference counting to be one of the possible algorithms of memory management. But many different programming environments have settled on the phrase "true garbage collection" to refer to a memory management scheme that can catch circular cycles. As a result of that I don't like calling reference counting "garbage collection".

On the question of which is better, I have been all over the map. I like reliable destruction mechanics - see my ReleaseAction for proof. But I've also been bitten by internal bugs from reference counting. I've had mod_perl systems whose memory consumption is higher than it should be in part because reference counting caused copy-on-write memory to be written (and therefore copied). I like using closure techniques that naturally lead to cycles and I've jumped hoops to avoid them.

My current preference is that I'd like to see true garbage collection, or else reference counting, and I'd like a hybrid system least. Not the least because I'd be afraid that the reference counting would not get maintained properly and would slowly degrade, while continuing to require a performance overhead.

Generally, unless it's supposed to run as a server (incoming requests can happen at any time) or start-up time is prohibitive, I prefer the cron-job solution myself anyway.

For example, with the CB stats, I have one app that's constantly running to watch IRC for CB messages. But that's because connecting to IRC and getting into the #cbstream channel can take a while, and I'll lose information if I'm not already in when the message occurs. However, everything else runs as cron jobs: pre-parsing data in the database (every 6 minutes) and creating the stats page (every 60 minutes).

The weak point really is that IRC application. I'm going to have to rewrite it at some point to us cb60 and/or the PM CB interface and gather its data via cron job, too, just for stability.

I've found some evidence of people complaining they run scripts and when the scripts exit,they do not release the memory they acquired back to the operating system

I didn't think that was possible, except maybe in DOS...

Devel::Leak will point you towards the objects that are not beeing gc'd. Once you know which objects might be causing the leak, you can use Devel::Cycle to watch those objects, and it will tell you if they contain any circular references.