Re: Tail, whiles and buffers ?

If your writing script is sleeping for one second between each iteration, there would be virtually no performance hit if you opened the file for output within the loop and closed it immediately afterward within the loop, so that it gets closed on each iteration just before sleeping. This would have the advantage of flushing the buffer on each iteration.

You should also implement flocking at both ends; flock your writes, and do a shared flock on your reads. There's no good reason that I can think of not to, given the examples you've posted.

Update:
An aside... Corion pointed out that on Win32 flock is unnecessary. So that part of the advice won't improve your situation if you're on Win32, but it won't hurt either other than adding a little code, and if the code may be executed in other OS's, it'll probably help.

At any rate, opening and closing the file within the loop should help.

Dave

Comment on Re: Tail, whiles and buffers ?

Replies are listed 'Best First'.
Re^2: Tail, whiles and buffers ? by Anonymous Monk on Oct 19, 2006 at 17:37 UTC
I think I found the answer...I was using a deprecated version of auto flush I guess. This seems to work: `use IO::Handle; open(WRITE, ">>C:\\File\\test.dat"); WRITE->autoflush(1);` [download]	[reply] [d/l]
Re^3: Tail, whiles and buffers ? by davido (Cardinal) on Oct 19, 2006 at 17:43 UTC
That's strange, your original post said nothing about using IO::Handle. So your example code was irrelevant. Nice. Dave	[reply]
Re^4: Tail, whiles and buffers ? by Anonymous Monk on Oct 19, 2006 at 17:47 UTC
Sorry about that Dave, It wasn't in my original post but after more research I tried it and it seemed to work.	[reply]
Re^4: Tail, whiles and buffers ? by Anonymous Monk on Oct 19, 2006 at 17:55 UTC
sorry Dave...my original post said I tried using "$\|++", hence the deprecated version of autoflush, so I did some more research and found that using the IO module works.	[reply]
Re^5: Tail, whiles and buffers ? by davido (Cardinal) on Oct 19, 2006 at 18:04 UTC
Re^5: Tail, whiles and buffers ? by blazar (Canon) on Oct 20, 2006 at 16:06 UTC
Re^2: Tail, whiles and buffers ? by Anonymous Monk on Oct 19, 2006 at 17:28 UTC
If I remove the sleep from my while loop you can see my issue. The tail script always seems to be several hundred lines behind.	[reply]
Re^3: Tail, whiles and buffers ? by davido (Cardinal) on Oct 19, 2006 at 17:44 UTC
Yes, but in your example code there was a sleep in the loop. Is that not relevant? Dave	[reply]
Re^2: Tail, whiles and buffers ? by Anonymous Monk on Oct 19, 2006 at 17:22 UTC
That does work, unfortunately my "real" script is waiting for output from a module that is using a foreach loop that could potentially contain thousands of elements, and I need to be able to detect them as soon as they get written.	[reply]
Re^3: Tail, whiles and buffers ? by davido (Cardinal) on Oct 19, 2006 at 17:46 UTC
Maybe writing to a file and reading from the file at the same time isn't the most efficient means of communication between the two scripts. It's possible to write to a logfile, and also open a pipe between two processes. With IO::Tee you could write to both filehandles simultaneously. Dave	[reply]