ursus has asked for the wisdom of the Perl Monks concerning the following question:

(cross-posted in comp.lang.perl.modules) Hey all, I'm looking for an elegant and (semi-) reliable language or module for filtering text streams. The 2 major features I'm looking for are multiplexing (input stream can be converted into several output streams) and syntactic simplicity. Any leads would be much appreciated.

Replies are listed 'Best First'.
Re: Stream processor?
by jdporter (Paladin) on Sep 24, 2007 at 21:57 UTC
    multiplexing (input stream can be converted into several output streams)

    If you're thinking in terms of unixy file i/o streams, you probably can't do better than a good modern shell, like ksh or bash. They let you open several (not sure what the limit is) streams for reading and writing very easily. See, for example, ksh93 i/o or bash i/o.

    A word spoken in Mind will reach its own level, in the objective world, by its own weight
Re: Stream processor?
by graff (Chancellor) on Sep 24, 2007 at 22:40 UTC
    The 2 major features I'm looking for are multiplexing (input stream can be converted into several output streams) and syntactic simplicity.

    If by "multiplexing" you mean that a single input stream would be (selectively?) directed to some number of output file handles or separate sub-processes, I don't see a need for anything beyond the standard "open()" call. Open as many output file handles as you need, whether they refer to disk files or to processes (using open my $fh, '|-', "process", @args). (Update: this sounds to me like it might be "de-multiplexing" (demuxing for short), which is sort of the opposite of "multiplexing" (muxing).)

    As for syntactic simplicity, you are either going to be doing a simple loop over file handles for each input record (print to each output in turn), or else you'll put the file handles into a "dispatch table" hash, and print to the ones you want according to whatever decision needs to be made on a record-by-record basis.

Re: Stream processor?
by mr_mischief (Monsignor) on Sep 25, 2007 at 05:41 UTC
    Is this along the lines of what you had in mind?
    #!/usr/bin/perl use strict; use warnings; my $i = 0; my @out_files; foreach ( qw( op0 op1 op2 op3 ) ) { unless ( open $out_files[ $i ], '>', $_ ) { die "Cannot open $_ for writing: $!\n"; } $i++; } while ( <> ) { foreach my $out ( @out_files ) { print $out $_; } }
    Or maybe something more succinct:
    #!/usr/bin/perl use strict; use warnings; my $line; my %out = map { $_ => undef } qw( op0 op1 op2 op3 ); ( open $out{ $_ }, '>', $_ or die "can't: $!\n" ) for keys %out; while ( $line = <> ) { print $_ $line for values %out; }
    Or perhaps this is "cleaner" for your taste:
    #!/usr/bin/perl use strict; use warnings; my @out = qw( op0 op1 op2 op3 ); my %out = map { $_ => undef } @out; for ( @out ) { open $out{ $_ }, '>', $_ || die "Can't: $!\n" } while ( my $l = <> ) { print $_ $l for values %out }
Re: Stream processor?
by ursus (Acolyte) on Sep 25, 2007 at 22:14 UTC

    Thanks for all the help. I think what I'm looking for hasn't been done. I'll post it as soon as it's semi-usable. I do have a followup question: Does XSLT necessarily produce XML? Or can it spit out arbitrary data (for my case character data is sufficient)?

    Yay the Monastery!