Re^3: Perl and memory usage. Can it be released?

GNU libc allocator is derived from Doug Lea malloc, a proven general-purpose allocator. Go on, unpack and read the source and the comments (I'm looking at glibc-2.17/malloc/malloc.c)

True, trims do not happen much because small data gets allocated from fastbins. But try to malloc a lot of somewhat larger blocks (couple hundred bytes each), and free them all. You shall see a shrink.

Update: from said malloc.c:

    DEFAULT_TRIM_THRESHOLD     128 * 1024
    DEFAULT_TOP_PAD            0
    DEFAULT_MMAP_THRESHOLD     128 * 1024
    DEFAULT_MMAP_MAX           65536
[download]

And please don't say "never". E.g. freeing a block 64k to 128k in size triggers fastbin consolidation. If your program has performed a work cycle, freeing all temps, then it is quite possible a trim takes place. It depends on usage.

Comment on Re^3: Perl and memory usage. Can it be released? Download Code

Replies are listed 'Best First'.
Re^4: Perl and memory usage. Can it be released? by bulk88 (Priest) on Feb 09, 2014 at 04:03 UTC
/* malloc_trim(size_t pad); If possible, gives memory back to the system (via negative arguments to sbrk) if there is unused memory at the `high' end of the malloc pool. You can call this after freeing large blocks of memory to potentially reduce the system-level memory requirements of a program. However, it cannot guarantee to reduce memory. Under some allocation patterns, some large free blocks of memory will be locked between two used chunks, so they cannot be given back to the system. The `pad' argument to malloc_trim represents the amount of free trailing space to leave untrimmed. If this argument is zero, only the minimum amount of memory to maintain internal data structures will be left (one page or less). Non-zero arguments can be supplied to maintain enough trailing space to service future expected allocations without having to re-obtain memory from the system. Malloc_trim returns 1 if it actually released any memory, else 0. On systems that do not support "negative sbrks", it will always return 0. */ [download] You will not get a trim unless you write proof of concept code specifically to get a trim. That malloc.c says threshold for mmap is alloc size of >= 1 MB. Your process memory space will not shrink in most usage.	[reply] [d/l]

Replies are listed 'Best First'.

Re^4: Perl and memory usage. Can it be released?
by bulk88 (Priest) on Feb 09, 2014 at 04:03 UTC

/*
  malloc_trim(size_t pad);

  If possible, gives memory back to the system (via negative
  arguments to sbrk) if there is unused memory at the `high' end of
  the malloc pool. You can call this after freeing large blocks of
  memory to potentially reduce the system-level memory requirements
  of a program. However, it cannot guarantee to reduce memory. Under
  some allocation patterns, some large free blocks of memory will be
  locked between two used chunks, so they cannot be given back to
  the system.

  The `pad' argument to malloc_trim represents the amount of free
  trailing space to leave untrimmed. If this argument is zero,
  only the minimum amount of memory to maintain internal data
  structures will be left (one page or less). Non-zero arguments
  can be supplied to maintain enough trailing space to service
  future expected allocations without having to re-obtain memory
  from the system.

  Malloc_trim returns 1 if it actually released any memory, else 0.
  On systems that do not support "negative sbrks", it will always
  return 0.
*/
[download]

[reply]
[d/l]