Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re^2: Suffering with buffering - how to wake 'less'

by benizi (Hermit)
on Jan 13, 2010 at 19:43 UTC ( [id://817263]=note: print w/replies, xml ) Need Help??


in reply to Re: Suffering with buffering - how to wake 'less'
in thread Suffering with buffering - how to wake 'less'

Wow. Cool.

I was worried it wouldn't work, since your test wasn't testing the same thing as mine. (The warnings were a side channel indication of the fact that 'less' was stopping 'perl'). But, it seems to do the trick. Thanks a bundle.

...Except now I'm confused. How does it work? What is buf(.pl) doing that prevents 'less' from stopping it? or that forces 'less' to keep reading? I'm kind of back to my initial question from another angle: what is 'less' doing to 'perl' that indicates to 'perl' that it should pause its output?

  • Comment on Re^2: Suffering with buffering - how to wake 'less'

Replies are listed 'Best First'.
Re^3: Suffering with buffering - how to wake 'less'
by BrowserUk (Patriarch) on Jan 13, 2010 at 19:59 UTC

    Once less has displayed the first screen full, it stops reading the pipe and waits for instructions from the keyboard. The pipe fills and won't accept anymore from the producer, so it blocks on the write until the pipe empties. Which it won't until you hit the space bar or similar.

    buf.pl uses a shared queue as a buffer between two threads, The read thread just keeps reading and pushing to the queue. The write thread reads from that queue and writes to the pipe. Because the two are asynchronous, when the write thread blocks because the pipe to less is full, the reader just carries on filling up memory until the producer finishes.

    The downside is that if the producer produces sufficient output, the perl process will eventually run out of memory and die.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      There it is, thanks. I was lost by the disconnect between the two facts: 1. 'less stops after one screen' and 2. 'perl produces more than one screenful before it pauses'.

Re^3: Suffering with buffering - how to wake 'less'
by ikegami (Patriarch) on Jan 13, 2010 at 20:40 UTC

    What is buf(.pl) doing that prevents 'less' from stopping it?

    Nothing. buf's output thread is still blocking at the same spot as the producer would without buf.

    less stops reading when it has enough to display what it needs to display. The producer doesn't know or care when that happens, so it keeps adding to the pipe until it fills up. Trying to add more to the pipe, the producer blocks until less starts emptying it.

    buf reads all that it can as fast as it can. That's likely to be faster than the producer can produce output, so the pipe from the producer never fills, so the producer never blocks. buf keeps the lines it read in a Queue from where the writer will fetch them when it unblocks.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://817263]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (1)
As of 2024-04-18 23:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found