in reply to AIX vs. Linux memory use

Your linux Perl is complied with threading enabled. I suggest you try recompiling to match AIX. Perl's memory management is quite different with threading enabled and I wouldn't be at all surprised to find that it still has a few leaks.

-sam

Replies are listed 'Best First'.
Re^2: AIX vs. Linux memory use
by scunacc (Acolyte) on Jan 29, 2007 at 18:50 UTC
    Dear Sam,

    If you look at the perl -V output I supplied for AIX, you'll see that the AIX one is already compiled with threads.

    (From the original post:

    Here's how things are built: AIX: ... ... osname=aix ... archname=aix-thread-multi ... usethreads=define ... useithreads=define ...

    )

    (I already have a larege multi-platform suite of modules and scripts that I developing and using and the environments are set up as nearly identically as possible between Linux and AIX already since many / most of the scripts are able to run on either platform. So, I appreciate the input, but that's not the issue here.)

    Kind regards

    Derek.

      Right you are - must have misread the AIX config lines. Still, if I were you I'd try dropping threads from the Linux config. Perhaps AIX's threading libs are more stable than Linux's.

      -sam

Re^2: AIX vs. Linux memory use
by glasswalk3r (Friar) on Jan 29, 2007 at 19:40 UTC

    Do you suggest that for non-thread environments that require high performance would be better to turn-off threading by compiling Perl from sources?

    Not that I'm a thread heavy user, but I read that Perl 5.8 support for threading is better than 5.6. Does it still have bugs?

    Alceu Rodrigues de Freitas Junior
    ---------------------------------
    "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill
      Dear Alceu,

      I don't *think* the overhead of compiling with threads on is a big issue for code that does not use threads. I am open to correction on that of course :-) I do have both a threaded and non-threaded available. Maybe I can test at some point.

      I've had to jump thru' hoops to get my *threaded* performance (I mean - in a genuinely threaded application) up to scratch though (bascially reducing the memory footprint per thread) because it clones the environment in the new thread, including loaded modules, and so on. But I do have threaded Perl apps that run with 100's of threads just fine.

      But, again, if you don't actually *use* threads, I'm not sure that matters.

      Kind regards

      Derek.

      Yes, I do recommend that. I haven't run any tests lately, but historically (early v5.8) threaded Perl has been slower than non-threaded Perl even when not using threads.

      -sam

        Well, I've narrowed it down to ......

        ...

        langof()

        in

        Lingua::Identify

        Called from

        Text::Compare
        If I set the lang to 'en' in Text::Compare's get_words() subroutine, it runs at a steady ~24MB on Linux. Of course, the metric is now different, since it's not "correctly" (as if it ever really was) identifying the languages, but for my app that's not really a problem, since I don't care. As long as it doesn't skew the results badly, - and since this isn't the only metric I'm using, it's not looking that bad.

        It's main effect is to cause the matches I get back to indicate more similarity than before. And, actually, I'd rather have more false positives - so it doesn't hurt too much.

        I'll talk to the Lingua people next I think now....

        Thanks for everyone's suggestions.

        Kind regards

        Derek.

        Dear Sam,

        Thanks - that's useful to know. Wasn't sure if that was true with ithreads or not. I'll test that. Any idea on %age?

        Kind regards

        Derek.