in reply to Serialization Format that can be read back in chunks?

Can't base64 (or even Math::Base85 ) do chunks?

Untested, and I don't remember what the 57 * 60 is all about. :-) Probably something to do with ideal chunk size for base64. 

#!/usr/bin/perl
use strict;
use MIME::Base64 qw( encode_base64 );

#encode
open INFILE, '<', $ARGV[0];
binmode INFILE;
open OUTFILE, '>', $ARGV[1];
my $buf;
while ( read( INFILE, $buf, 60 * 57 ) ) {
    print OUTFILE encode_base64($buf);
}

close OUTFILE;
close INFILE;

###################################################
#decode_base64.pl:
#!/usr/bin/perl
use strict;
use MIME::Base64 qw( decode_base64 );

open INFILE,  '<', $ARGV[0];
open OUTFILE, '>', $ARGV[1];
binmode OUTFILE;
my $buf;
while ( $buf = <INFILE> ) {
    print OUTFILE decode_base64($buf);
}

close OUTFILE;
close INFILE;

[download]
I'm not really a human, but I play one on earth. Cogito ergo sum a bum

Replies are listed 'Best First'.
Re^2: Serialization Format that can be read back in chunks?
by pKai (Priest) on Mar 29, 2007 at 22:10 UTC
    57 * 4 / 3 = 76

    the max length for a base64 coded line required by rfc2045 for MIME encoded data (mails).

    No idea if the 60 has any hard technical background though.

    Anyway, I don't know if such a character level encoding is the solution to Mike's problem. After all, this is completely unaware of the structure of the data encoded.

    Serializer/Formats as Data::Dumper, XML, YAML generate output which can be described by contextfree languages, which probably is no coincidence. Resorting to the pumping lemma the maximal distance between the start/end braces/tags/indentation-changes corresponding to the start/end of an embedded serialized substructure is not limited, which sounds quite incompatible to "chunk" (i.e. fixed-length) to me.

[reply]
[d/l]

In Section Seekers of Perl Wisdom