in reply to Re^9: Consumes memory then crashs
in thread Consumes memory then crashs

Don't fret over the couple of kilobytes of crap® you had to download. I installed a whole Strawberry Perl just to test your code on Windows :)

Don't blame Perl or threads because your C runtime libraries are paying lip service to the realities of concurrency.

Assuming this to be the case, I have bad news for you: so does Windows' C library. Here's a snippet of your "fixed" code's output on the latest Strawberry on XP---the runtime was just as atrocious as on Linux BTW:

$ uniq -c outfile |head -n10
    819 Thread 2
      1 ThThread 1
    818 Thread 1
      1 ThThread 3
    818 Thread 3
      1 ThThread 4
    818 Thread 4
      1 ThThread 5
    818 Thread 5
      1 ThThread 6

[download]

At Thread 20 it gets a bit more irregular ...

If it doesn't take care of things, you'll have to do it.

No, you'll have to do it. If you want to show it can be done correctly using threading that is.

The addition of my $old = select $fh; $|++; select $old; might sort out the interleaving problem.

That doesn't sound overly confident ...

And what is the point of running 500 threads?

Contrary to what you wrote, a thread is not just a means of improving efficiency on multiprocessors. It's just a logical program flow that's supposed to correspond to a certain task, and there are plenty of tasks that could make use of hundreds of threads: simulations, network monitoring, sensor data collection, crawling slow sites, etc. If threading in the "use threads" sense was halfway efficient for it that is.

It takes ~40MB to run 4 threads. That's a whole 2% of the ram of the lowest spec commodity box you'll ever find for sale.

You still get 1 GB netbooks, but whatever. So you need only twice the memory to get correct results most of the time, with nondeterministic failures in-between, after the first "Service Pack". At least the other side's IIS doesn't seem to scale either so you don't have to put the thread code to this test. Sorry, even though I have conceded from the start that there are rare cases where threads are the model of choice this is not gonna convince me that we're looking at one.

As for POE on Windows, on my fresh Strawberry install I can type "cpan POE" and then "cpan POE::Component::Client::HTTP" to end up with a working installation of everything required. Don't blame POE when ActiveState's repository absorbs excrement.

One way uses 0.5% of my memory; only needs what came installed with my Perl installation; and works. The other two require gobs of extra code and either don't run or never finish. The decision is an easy one for me.

No, it does not work, at least not reliably. That was the whole point. The failure rate may be acceptable for you but I don't think it's "a simple solution to thread this properly" as the OP asked.

Replies are listed 'Best First'.
Re^11: Consumes memory then crashs
by BrowserUk (Patriarch) on Mar 26, 2012 at 03:56 UTC
    on the latest Strawberry

    Ah! That's built with the gcc runtime isn't it!

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
      on the latest Strawberry
      Ah! That's built with the gcc runtime isn't it!

      Yes. If you insist, here's some uniq'd output from a freshly installed ActivePerl 5.14.2 which they tell me is built with VisualStudio:

      $ uniq -c outfile | head -n25
    682 Thread 2
      1 Thread 2Thread 3
    681 Thread 3
      1 Thread 3Thread 4
    681 Thread 4
      1 Thread 4Thread 5
    681 Thread 5
      1 Thread 5Thread 1
    681 Thread 1
      1 Thread 1Thread 6
    681 Thread 6
      1 Thread 6Thread 7
    681 Thread 7
      1 Thread 7Thread 8
    681 Thread 8
      1 Thread 8Thread 9
    681 Thread 9
      1 Thread 9Thread 10
    629 Thread 10
      1 ThThread 11
    629 Thread 11
      1 ThThread 12
    629 Thread 12
      1 ThThread 13
    629 Thread 13

      [download]

      If Perl's threading primitives' semantics were broken unless compiled with a runtime that <<10% of the installed Perls use, I'd still call Perl broken. If I can reproduce the bug on each an every Perl I look at though nobody seems to have complained about the primitives you use being buggy, chances are your Perl code is broken.

      I don't know which it is, but it remains that I haven't seen a reliable threaded implementation yet. I for one would centralize the printing and use another Queue to feed back the results from the threads as at least the queue locking seems (as far as I've looked which is not very deep at all) to be working. But then again, why? I don't get the resistance against POE which fits on a floppy disk and is some 20kLOC without POD when common threading implementations are on the same order of magnitude (of rather more complex C code at that) and 20 MB of RAM are insignificant to you.

[reply]
[d/l]

        This is your 500-do-nothing-useful-threads-as-fast-as-you-can-with-no-attempt-to-solve-either-the-OPs-problem-or-work-out-what-you-are-doing-wrong code.

        Sorry, but I have neither time nor inclination to fence with someone who's purpose is anything but. to help.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?
[reply]

In Section Seekers of Perl Wisdom