in reply to Memory has been increasing, unable to release memory.

There are several issues with your code:

Please read How do I post a question effectively?

It looks like you want to call another perl program on a list of files. You should turn that program into a module, with one exported function, and call that function on your files, rather than spawn a new process with system. You'll find some information on modules in the perlmod documentation and in tutorials like this one

  • Comment on Re: Memory has been increasing, unable to release memory.

Replies are listed 'Best First'.
Re^2: Memory has been increasing, unable to release memory.
by Perl_Love (Acolyte) on Mar 27, 2017 at 15:45 UTC

    Hi Eily . thank you for such a detailed answer!

    Actually, I wrote the two script with the fork.pl call another crawler.pl .The two script memory will keep increasing, I suspect I wrote the script with some problem, the reference count not zero, so the memory is not released.

    $fork, I placed in sub Config, $fork=$json_conf->{'fork'}, I set up 500 in the conf.json file.

    undef, I tried to release the memory, really no effect !

    Two while (1): are a crawler script, the first while (1), I let it keep running. While the second while (1), because I found that sometimes get error (it seems that SSL error or other things what I forget) the script will be directly out.So I have a idea with eval, it can work, although it is not perfect, let me change it again :)

    You suggest to changed script to modules, whether should also solved memory surge problem ,so that it will have effect ?

    I would like to use the Devel:: Peek module, but 5.20.3 version can not install, I do not know how to analyze the problem of memory surge, please help me to check Thanks a lot!

      Anonymonk's answer seems to point out a good candidate for your memory problem. My suggestions were rather means to make your code easier to read and understand, which would make debugging easier. I'm sure that creating so many independant perl processes doesn't help though. Adding use strict; and use warnings; can help you solve your memory issues though, as it points out potential mistakes and known pitfalls.

      And you're right about $fork, I did not see that it is indeed set. But this demonstrates that your code is hard to read. By the way, the & isn't necessary to call a function, instead, if you want to make a call more explicit, you can use parentheses, like this: Config();.

        Hi~

        My other question:

        http://www.perlmonks.org/?node_id=1165988

        afoken reply about strict, I add use strict now.

        $fork.pl set max 500 fork, and all file is 300.

        Half the file called crawler.pl, and the other half of the file has been running in fork.pl.

        print account, I can't find anything problem

        Maybe I need to rewrite the whole program

      You suggest to changed script to modules, whether should also solved memory surge problem ,so that it will have effect ?
      It might help with some problems, and it might make other problems worse. Unfortunately, leaking memory is one of the problems that might be made worse. In most cases, I would tend to agree with Eily's advice, but it's probably not a high priority for you.

        delete system("/home/root/Perl/zhubo/crawler.pl $_");

        add sleep 11111;

        Problems exist