comment on

I'll apologise now for the length of this one, but I'm in need of some serious monkery.

I've created a script that reads in several files, splitting each line into separate fields, and storing them all in a hash, to then roll around these hashes to produce an output file. All works well (for once).

The problem comes when I try and implement this on a new box. Under AIX 5.1 it all works fine, under AIX 5.3 it kills the process with 'out of memory'. Both boxes have 1GB of RAM and 2GB swap space - should be more than enough.

Using a cut down set of input files, the working box uses about 53MB of memory to run this job, the new one uses just over 1GB.

This points to altered memory usage, and indeed the build options are different between the two boxes. The trouble is that I have no idea what might cause this.

Any thoughts welcome.

UPDATE: - upgrading to 5.8.8 did indeed cure the problem On the Old box, perl -V gives:

Summary of my perl5 (revision 5.0 version 6 subversion 0) configuratio
+n:
  Platform:
    osname=aix, osvers=5.0.0.0, archname=aix
    uname='aix shaq 1 5 006044854c00 '
    config_args='-de'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultipl
+icity=undf
    useperlio=undef d_sfio=undef uselargefiles=define 
    use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=u
+ndef
  Compiler:
    cc='cc', optimize='-O', gccversion=
    cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=
+16384'
    ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=
+16384 -q3'
    stdchar='unsigned char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksiz8
    alignbytes=8, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='ld', ldflags ='-b32'
    libpth=/lib /usr/lib /usr/ccs/lib
    libs=-lbind -lnsl -lgdbm -ldbm -ldb -ldl -lld -lm -lC -lC_r -lc -l
+crypt -lbv
    libc=/lib/libc.a, so=a, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -bE:/usr/
+opt/perl5'
    cccdlflags=' ', lddlflags='-bhalt:4 -bM:SRE -bI:$(PERL_INC)/perl.e
+xp -bE:$('


Characteristics of this binary (from libperl): 
  Compile-time options: USE_LARGE_FILES
  Built under aix
  Compiled at Nov 22 2000 08:49:49
  @INC:
    /usr/opt/perl5/lib/5.6.0/aix
    /usr/opt/perl5/lib/5.6.0
    /usr/opt/perl5/lib/site_perl/5.6.0/aix
    /usr/opt/perl5/lib/site_perl/5.6.0
    /usr/opt/perl5/lib/site_perl
    .
[download]

On the new box:

Summary of my perl5 (revision 5.0 version 8 subversion 2) configuratio
+n:
  Platform:
    osname=aix, osvers=5.2.0.0, archname=aix-thread-multi
    uname='aix perlfly 2 5 000ad7df4c00 '
    config_args=''
    hint=previous, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=de
fine
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc_r', ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURC
+E -qmaxmem
=16384 -qnoansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -q32 -D_L
+ARGE_FILES
 -qlonglong',
    optimize='-O',
    cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=
+16384 -qno
ansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -D_ALL_SOURCE -D_ANS
+I_C_SOURCE
 -D_POSIX_SOURCE -qmaxmem=16384 -qnoansialias -DUSE_NATIVE_DLOPEN -DNE
+ED_PTHREAD
_INIT -q32 -D_LARGE_FILES -qlonglong -D_ALL_SOURCE -D_ANSI_C_SOURCE -D
+_POSIX_SOU
RCE -qmaxmem=16384 -qnoansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_IN
+IT -q32 -D
_LARGE_FILES -qlonglong -D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE
+ -qmaxmem=
16384 -qnoansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -q32 -D_LA
+RGE_FILES 
-qlonglong -D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=163
+84 -qnoans
ialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -q32 -D_LARGE_FILES -ql
+onglong -D
_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=16384 -qnoansial
+ias -DUSE_
NATIVE_DLOPEN -DNEED_PTHREAD_INIT -q32 -D_LARGE_FILES -qlonglong -D_AL
+L_SOURCE -
D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=16384 -qnoansialias -DUSE_NAT
+IVE_DLOPEN
 -DNEED_PTHREAD_INIT -q32 -D_LARGE_FILES -qlonglong'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksize
=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='ld', ldflags =' -brtl -b32 -bmaxdata:0x80000000'
    libpth=/lib /usr/lib /usr/ccs/lib
    libs=-lbind -lnsl -ldbm -ldl -lld -lm -lpthreads -lc_r -lcrypt -lb
+sd -lPW
    perllibs=-lbind -lnsl -ldl -lld -lm -lpthreads -lc_r -lcrypt -lbsd
+ -lPW
    libc=/lib/libc.a, so=a, useshrplib=true, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags='-bE:/usr/op
+t/perl5/li
b/5.8.2/aix-thread-multi/CORE/perl.exp -bE:/usr/opt/perl5/lib/5.8.2/ai
+x-thread-m
ulti/CORE/perl.exp -bE:/usr/opt/perl5/lib/5.8.2/aix-thread-multi/CORE/
+perl.exp -
bE:/usr/opt/perl5/lib/5.8.2/aix-thread-multi/CORE/perl.exp'
    cccdlflags=' ', lddlflags='-bhalt:4 -bM:SRE -bI:$(PERL_INC)/perl.e
+xp -bE:$(B
ASEEXT).exp -bnoentry -lpthreads -lc_r'


Characteristics of this binary (from libperl): 
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL
+_IMPLICIT_
CONTEXT
  Built under aix
  Compiled at Feb 13 2004 13:18:17
  @INC:
    /usr/opt/perl5/lib/5.8.2/aix-thread-multi
    /usr/opt/perl5/lib/5.8.2
    /usr/opt/perl5/lib/site_perl/5.8.2/aix-thread-multi
    /usr/opt/perl5/lib/site_perl/5.8.2
    /usr/opt/perl5/lib/site_perl
    .
[download]

'I think the problem lies in the fact that your data doesn't fit my program'.

In reply to Hashing Memory Usage by awkmonk

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.