syswrite and STDOUT

OverlordQ has asked for the wisdom of the Perl Monks concerning the following question:

Greetings monks! After playing with some flames and shoop, I was wanting to do some post processing to the mpeg's that shoop creates, but due to a 'magical header' it causes most editing programs to choke on it, so I borrowed this code from the flam3 mailing list. Only problem is, when it attempts to syswrite I get:

Broke: Not enough space

It works if I print to a normal filehandle, but I'm wanting to pipe the output to another program.

#!/usr/bin/perl -w

use strict;
use File::Spec;

my $cwd;
my $dir;

my $fname;
my $buffer;
my @stats;

chomp ($cwd = `cd`);

# read and use directory
$dir = <>;
chomp $dir;
$dir =~ s/\r//;

# read sheep
while ($fname = <>) {
    # remove trailing newline
    chomp $fname;
    # remove trailing CR (windows only)
    $fname =~ s/\r//;

    $fname = File::Spec->catfile($cwd, $fname) or die "Can't cat: $!\n
+";
    @stats = stat ($fname) or die "Can't stat $fname $!\n";

    open(INFILE,"< $fname") or die "Can't open $fname $!\n";
    binmode INFILE;
    read(INFILE, $buffer, $stats[7]) or die "Broke: $!\n";
    close INFILE;

    $buffer =~ s/(.====magic-header====.*$)//s;
#    open(OUTFILE,">outfile.mpg") or die "Broke: $!\n";
#    syswrite(OUTFILE, $buffer) or die "Broke: $!\n";
    syswrite(STDOUT, $buffer) or die "Broke: $!\n";
#    close(OUTFILE);
}
[download]

Comment on syswrite and STDOUT Download Code

Replies are listed 'Best First'.
Re: syswrite and STDOUT (nibble) by tye (Sage) on Apr 29, 2005 at 06:35 UTC
Sounds like you are trying to write too big of a buffer (for the pipe). `while( 4096 < length $buffer ) { syswrite( STDOUT, $buffer, 4096 ); substr( $buffer, 0, 4096, '' ); } syswrite( STDOUT, $buffer );` [download] Perhaps? - tye	[reply] [d/l]
Re: syswrite and STDOUT by polettix (Vicar) on Apr 29, 2005 at 09:05 UTC
I believe that tye is on the correct path, but I'm curious about the system you're on. `syswrite()` is supposed to return the number of bytes actually written and not chocke if there are too much of them. Also, note that `syswrite()` could return 0 without implying an error (which is signaled by `undef`). I believe that Linux `write(2)` actually works like this. AFAIK, POSIX requires the minimum buffer size for pipes and stuff like that to be at least 512. Most modern system should support greater buffers, and this is why tye uses 4096 (which I remember to be the default size in Linux). To be on the safest side I'd stick to 512, but you could be on a non-POSIX system and need to adjust your aim :) Flavio (perl -e 'print(scalar(reverse("\nti.xittelop\@oivalf")))') Don't fool yourself.	[reply] [d/l] [select]
Re^2: syswrite and STDOUT (guess) by tye (Sage) on Apr 29, 2005 at 15:28 UTC
Win32 was mentioned in the CB. I suspect that reporting $^E instead of just $! might be more instructive. But it isn't just the buffer size of the pipe (which defines how much can be sent atomically); I'm assuming that the amount of data is so much that the kernel exhausted some resource trying to buffer it all. I find it surprising that this would happen (and I doubt it would on Unix), but that was the best guess I could come up with. - tye	[reply]
Re^3: syswrite and STDOUT (guess) by polettix (Vicar) on Apr 29, 2005 at 15:41 UTC
But it isn't just the buffer size of the pipe (which defines how much can be sent atomically); I'm assuming that the amount of data is so much that the kernel exhausted some resource trying to buffer it all. You're right, I merged the concepts, even if some lurking inside the 2.4.x Linux kernel convinced me that Linux considers them to be the same, as far as I can remember. Anyway, the fractioned approach you suggest should be used more generally - testing if `write/syswrite` actually wrote all requested characters should become an habit much like testing if `open` was successful. Flavio (perl -e 'print(scalar(reverse("\nti.xittelop\@oivalf")))') Don't fool yourself.	[reply] [d/l] [select]