in reply to perl 5.12.3 + threads + memory leak

http://rt.perl.org/rt3/Ticket/Display.html?id=69598 It is fixed in 5.13, but not in 5.12.

As alternative, I can recommend AnyEvent::HTTP, which on my taste is much better than using threads.

  • Comment on Re: perl 5.12.3 + threads + memory leak

Replies are listed 'Best First'.
Re^2: perl 5.12.3 + threads + memory leak
by kamenpetrov (Novice) on Feb 15, 2011 at 10:45 UTC

    Thanks for the info.

    I have tested the following scenarios:
    - 5.10.0/5.12.3 + LWP --> both LWP and Perl leaks memory
    - 5.10.0/5.12.3 + curl --> Perl leaks memory
    - 5.13.9 + LWP --> LWP leaks memory
    - 5.13.9 + curl --> no leak


    Thank you once again for your time and help!
      Ok, now to the real problem:
      - perl 5.13.9
      - proc::daemon
      - threads
      - DBD::Pg

      So, I have a script running as a daemon. The daemon listen to a Postgres database for "notify" events. Once there is new event, the daemon fetch all records from a database table, passes them back to a database procedure (equal to SELECT statement) and the result is passed as string to another sub routine in the perl script. The routine generates an URL and that URL is fetched via curl in a perl thread.

      All that works good, except the fact DBD::Pg leaks memory. All the code is "strict"/"warnings"-safe and the Pg module leaks really serious amount of memory. The module I pick for tracing the leaks returns many records in the DBI.pm. Looking trough DBD::Pg code, I noticed the CLONE function does not really do what it is supposed to do (compare to DBD::mysql). It only exists so to stop the DBI module from reporting warnings.

      Does anyone of you guys have used DBD::Pg in a multi-threaded script and how you manage the memory leaks?

      Thanks

        In general--I'm having to be general because I don't use *nix, and these leaks do not happen on my system--thread leaks only become a problem if you create and destroy threads in large numbers. And there is rarely ever a reason to do so. It is almost always better to start one or a pool of threads and reuse them.

        It's not clear from your description which parts of your application your are theading, which makes it difficult to offer specific solutions. If you could post the code, it would be much easier to help you.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.