Re^2: OS memory reclamation with threads on linux

I watch the %MEM columns in top/ps. It's been confusing testing, because when there is a small percent difference between the peak and normal mem usage, sometimes the kernel shows %MEM drop, other times not. If the diff is big, the kernel immediately returns it.

My real question is what are people talking about when they claim their threaded apps are increasing in memory. Do I just say "forget it.... the kernel will grab it if needed", or "you have a thread design problem"

I'm also wondering if this has different behavior with the older 5.8 threads vs. the latest threads (which seem to run better), and/or kernel versions.

It seemed like just last week that this was a constant problem with Perl threads, and now suddenly it seems like a great thing, where you can reclaim memory from mem-intensive Perl scripts, by putting the mem-intensive stuff into a thread.

I'm not really a human, but I play one on earth Remember How Lucky You Are

Comment on Re^2: OS memory reclamation with threads on linux

Replies are listed 'Best First'.
Re^3: OS memory reclamation with threads on linux by jbert (Priest) on Aug 27, 2008 at 19:58 UTC
I watch the %MEM columns in top/ps Well, that's proportional to RES, it's the amount of physical memory used by the app at that moment (including all loaded libs and possibly shared memory). It only reflects your apps "memory usage" if the box isn't under memory pressure (i.e. too much running at once). If your box is under memory pressure, then RES and %MEM will relate to what the OS is running at that moment - that figure will go down when bits of your app is paged out for something else to run, even though your app is still "using the same amount of memory" (i.e. it hasn't free'd anything). My real question is what are people talking about when they claim their threaded apps are increasing in memory. My first guess would be that they've just got a good old-fashioned memory leak. The OS generally makes good decisions regarding memory use. The best advice is to not worry about it too much until you have a problem app, unless you're into learning things (which is cool).	[reply]

Replies are listed 'Best First'.

Re^3: OS memory reclamation with threads on linux
by jbert (Priest) on Aug 27, 2008 at 19:58 UTC

I watch the %MEM columns in top/ps

Well, that's proportional to RES, it's the amount of physical memory used by the app at that moment (including all loaded libs and possibly shared memory). It only reflects your apps "memory usage" if the box isn't under memory pressure (i.e. too much running at once). If your box *is* under memory pressure, then RES and %MEM will relate to what the OS is running at that moment - that figure will go down when bits of your app is paged out for something else to run, even though your app is still "using the same amount of memory" (i.e. it hasn't free'd anything).

My real question is what are people talking about when they claim their threaded apps are increasing in memory.

My first guess would be that they've just got a good old-fashioned memory leak.

The OS generally makes good decisions regarding memory use. The best advice is to not worry about it too much until you have a problem app, unless you're into learning things (which is cool).

[reply]