Re: A way to see how much 'memory' a program will take up in, say, mod

It sounds like you are asking to predict the memory that will be used by an application, right?

Is the application data driven? In other words, the data passed into an application can cause the size used to grow tremendously. As a simple example...

while (<>) {
  push(@lines, $_);
}
[download]

--MidLifeXis

Comment on Re: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? Download Code

Replies are listed 'Best First'.
Re^2: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by MashMashy (Sexton) on Feb 19, 2009 at 18:54 UTC
You know, you are absolutely right - it does read in from a SQL database, which makes this much more complicated.. How would you suggest going about something like 'cutting down the amount of memory taken in mod_perl?'	[reply]
Re^3: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by MidLifeXis (Monsignor) on Feb 19, 2009 at 19:08 UTC
Profile, then find the worst offending areas, and target them. Perhaps a better data structure, or a different algorithm would change the usage. Update: Perhaps one of the Devel::Size type modules might help with this. Some algorithms have more memory overhead but run faster. Others run more slowly, but take less memory. A contract I am doing right now has frozen data in a database (basically a key/value setup with a frozen representation of a HoHoHoH... structure as the value), Some of the values are causing the application to run out of memory when being processed. By making each level of the hash another column and letting the database do much of the heavy lifting, I am saving memory, hopefully saving time (not needing to thaw the large data structure), and making a much more scalable application. In summary, the algorithm and data structure can make a tremendous difference. --MidLifeXis	[reply]
Re^4: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by MashMashy (Sexton) on Feb 19, 2009 at 19:23 UTC
Se, I profiled it, and it runs nice and fast - but my issue is, it takes up a lot of memory, which with enough processes becomes swap, which becomes crash. dprofpp doesn't say what memory it takes up - is there anything that does?	[reply]
Re^3: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by perrin (Chancellor) on Feb 19, 2009 at 19:10 UTC
There's a ton of documentation on this subject at http://perl.apache.org and http://modperlbook.com.	[reply]
Re^4: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by MashMashy (Sexton) on Feb 19, 2009 at 19:24 UTC
See, I've read both of those - one cover-to-cover (in dead-tree format) - but while they talk a lot about how to optimize mod_perl, there's nothing I've found that says 'type this to see how much memory your program will take up'.	[reply]
Re^5: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by perrin (Chancellor) on Feb 19, 2009 at 20:28 UTC
Re^3: A way to see how much 'memory' a program will take up in, say, mod_perl, so I can start to optimize it? by zwon (Abbot) on Feb 19, 2009 at 22:05 UTC
So your program makes some SQL requests? Maybe returned dataset is too big? Test your SQL requests in standalone scripts and see how much memory they would use.	[reply]