kulls has asked for the wisdom of the Perl Monks concerning the following question:

Greetings,
Assume, I'm trying to push 6000 records to an array. Eventhough If you are enable  "$|=1 " the buffer,I still believe it wait to complete the loop.
Please suggest me, How we can display the data as soon as it receives the data.
-kulls

Replies are listed 'Best First'.
Re: flushing the data from array
by kyle (Abbot) on Jan 14, 2008 at 18:49 UTC

    I don't understand your question. Can you show us the code that's causing you a problem (along with some idea of what you expect it to do, and how what it's doing now differs from that)?

      Hi,
      Thanks for your reply.
      Here is a sample code.
      #!/usr/bin/perl my (@array,$r); for(1..5000000) { $r=($_ * 100 + 300 + 200); push @array, $_; } return \@array;
      I guess we need to wait untill the loop finish the process and entire data in the memory. Instead of doing this, Is there any ways to return the array data then and there. Did you get my point ?
      -kulls

        No, I don't think I get your point. You're doing a lot of looping and returning an array that you create in the loop. You can't return the array during the loop because it's not actually finished yet.

        Your original post mentions $|=1 (aka $OUTPUT_AUTOFLUSH if you use English). This is for sending output (as with print) without buffering (see perlvar). If your loop looks like this:

        $| = 1; # output autoflush for ( 1 .. $x ) { push @array, $_; print "I did '$x'! Yay!\n"; }

        ...then in that case, $| makes a difference. If you're not actually generating any output during the loop, it won't matter. The $| setting is only for output.

        If you leave $| to the default value (buffering), you'll get groups of lines at a time instead of each individually. Again, none of this has anything to do with looping or returning data.

        kulls:

        Your array data doesn't exist until the loop completes.

        Perhaps you mean you want to remove the explicit loop? If so, you might try using map to generate your series in a single statement. However, you'll still have your memory problems.

        Or maybe you're interested in a continuation? That is, a function that you can just get the next value from when you want it? That would let you work with your data without having to store it in an array.

        #!/usr/bin/perl -w use strict; use warnings; # Build a continuation to return the series of interest sub make_a_continuation { # Variables the continuation uses my $max = shift; my $first = 1; # Return an anonymous function ref to generate next item return sub { $first++ * 100 + 300 + 200; } } # Build a couple continuations my $get_next = make_a_continuation(50); my $get_another = make_a_continuation(100); # None of the data elements in the two series exists yet! You'll # have to create them as you need them by calling the appropriate # continuation, e.g. &$get_next. for my $loop (1..100) { # Randomly create & print items from the series print &$get_next . "\n" if rand(5) < 2.0; print &$get_another . "\n" if rand(5) < 3.5; }
        A test run on my machine here gives me:

        $ ./continuation.pl Next: 600 Another: 600 Another: 700 Next: 700 Next: 800 Another: 800 Next: 900 Another: 900 Another: 1000 Another: 1100 Next: 1000
        ...roboticus

        I personally believe that in addition to what that has already been said, there are some issues with your code:

        • it is a supposedly minimal snippet but it contains an unnecessary assignment to $r, which is not used afterwards;
        • it contains a bare return outside of a sub which is a valid construct, but a suspect one as well, and not likely to do what that you want.
        I guess we need to wait untill the loop finish the process and entire data in the memory. Instead of doing this, Is there any ways to return the array data then and there. Did you get my point?

        Not at all... do you want to randomly return asynchronously copies of your array while it is being filled? It should be doable with threads, but... why so?!?

        --
        If you can't understand the incipit, then please check the IPB Campaign.
Re: flushing the data from array
by GrandFather (Saint) on Jan 14, 2008 at 18:57 UTC

    High level arrays have little or nothing to do with buffering and displaying data. Perhaps you need to show us a little code that demonstrates your issue. Generate a minimal runable sample that shows the issue and remember to describe what you see and what you expect.


    Perl is environmentally friendly - it saves trees
Re: flushing the data from array
by aquarium (Curate) on Jan 15, 2008 at 01:29 UTC
    if you want a good answer, you need to be careful to phrase the question well. the original question talks of 6000 records, but then the example uses 5 million records.
    as everybody has been trying to get across...the buffering in perl, and in all computer languages and operating systems that support buffering, has to do with input/output, and absolutely nothing to do with memory structures.
    also the example code creates a rather useless (huge) array of elements that can be computed on the fly, without ever creating an array in the first place. was this the intention of the question?
    the hardest line to type correctly is: stty erase ^H