After a long long time fussing, fiddling, tweaking, more than a few total rewrites, and lots of learning (in a big part here), I've finally released my Data::Dumper replacement Data::Dump::Streamer v1.0 hit CPAN last night, and v1.01 with a bunch of doc fixes and enhancements was just uploaded a few minutes ago. (Please refrain from installing v1.0 as the documentation in it is incorrect in a few ways that will probably just annoy you :-)

There always been a lot of talk in the monastery about reinventing wheels. When and when not to, why not to etc. Since Data::Dumper is a pretty serious wheel, I feel I should offer the following justification for reinventing it.

Some people might ask why I didn't focus on making Data::Dumper better and resolving these problem with it. My answer is a mixture of issues with the API of Dumper, the basic mechanism of how it dumps, and the fact that any changes need to be made in both C and Perl. Also, I don't consider Data::Dumper in any way obsoleted by my module. For small to medium size simple structures its hundreds of times faster and perfectly accurate. But if you are dealing with hairy structures, or if you need to read the results of a complex structure, then I think you should try Data::Dump::Streamer out.

Now I have to take the earlier not-so-great version off CPAN. :-)

Much thanks to broquaint for the help on the module over its lifetime, and all of the others who helped test or suggested stuff. Cheers to the Monastery for the being a great place to learn. :-)


---
demerphq

    First they ignore you, then they laugh at you, then they fight you, then you win.
    -- Gandhi


Replies are listed 'Best First'.
Re: Introducting Data::Dump::Streamer
by halley (Prior) on Feb 24, 2004 at 15:33 UTC

    About two years ago, I made a post-processing filter for Data::Dumper which would make the output more legible. Not as radical as going breadth-first instead of depth-first, but it helped format the output much more readably for human consumption.

    As with you, I found the C-and-Perl issue to be daunting. If not in skills required, then in hassle to overcome. So I just decided to not bother. I would have to rework the main loop of Dumper to facilitate the filtering, and I just couldn't be arsed to jump through that hoop.

    It's also part of the main distribution, now, so any changes have to be checked just that much more deeply. It's basically frozen in time and won't get much better than it is. Which is a shame.

    --
    [ e d @ h a l l e y . c c ]

Re: Introducting Data::Dump::Streamer
by jplindstrom (Monsignor) on Feb 24, 2004 at 20:12 UTC
    Regarding the Windows memory issue (if someone wants to stay with Data::Dumper): it's not nearly as bad if you force it to use the pure-Perl implementation by setting the $Data::Dumper::Useqq flag. Without that, I've had data structures taking up hundreds of megabytes of RAM.

    Your module sounds great though.

    /J

Re: Introducing Data::Dump::Streamer
by demerphq (Chancellor) on Feb 25, 2004 at 10:29 UTC

    Id just like to address a couple of minor points here. There was some chattering yesterday in the CB that Data::Dump::Streamer fails to dump

    my @a = ('a0'..'a9'); unshift @a, \$a[2]; print Dump \@a;

    corectly, in a similar way to modules like YAML, Data::Dumper, Data::Dump fail this test. I assure you that this is NOT the case:

    $ARRAY1 = [ 'R: $ARRAY1->[3]', 'a0', 'a1', 'a2', 'a3', 'a4', 'a5', 'a6', 'a7', 'a8', 'a9' ]; $ARRAY1->[0] = \$ARRAY1->[3];

    which is perfectly correct. A thorough examination of this problem can be found in the dogpound.t test file contained in the distribution.

    The other thing is that there seem to be difficulties installing the module on some architectures and version of perl. Ive tested as well as I can on Win32 and some versions of linux, but obviously cant test everywhere. Please email me the results of TEST_VERBOSE=1 if you have a failed build.

    UPDATE: grinder kindly obliged me with a test output from a failed build, (thanks dude!) and it looks like its a bug in perl. If the only failure is test 14 of dump.t (all the reports so far are for this test) and you get the following as part of the verbose test output then your perl is buggy in that (1) x 4 returns four identical objects and not four seperate objects. (Or is it the other way round and the perls that _dont_ do this are buggy?, either way there are two distinct behaviours out there and Streamer notices the difference. :-)

    Rle(1) Alias Got: $ARRAY1 = [ 1, 'A: $ARRAY1->[0]', 'A: $ARRAY1->[0]', 1, 0, 'A: $ARRAY1->[3]', 1, 'A: $ARRAY1->[6]', 'A: $ARRAY1->[6]' ]; make_ro($ARRAY1->[0]); alias_av(@$ARRAY1, 1, $ARRAY1->[0]); alias_av(@$ARRAY1, 2, $ARRAY1->[0]); make_ro($ARRAY1->[4]); alias_av(@$ARRAY1, 5, $ARRAY1->[3]); make_ro($ARRAY1->[6]); alias_av(@$ARRAY1, 7, $ARRAY1->[6]); alias_av(@$ARRAY1, 8, $ARRAY1->[6]); EXPECT

    Thanks and cheers.


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi


Re: Introducting Data::Dump::Streamer
by jonadab (Parson) on Feb 25, 2004 at 02:23 UTC

    One limitation I've noticed in Data::Dumper is that it doesn't handle closures. e.g.,

    my $foo; { my $bar = 'bar'; $foo = sub { return ($bar .= join $bar, @_) } } use Data::Dumper; print Dumper(\$foo);

    That doesn't work. At all. (Okay, it prints *something*, but it's not anything *useful*.) Does your module handle this better? Or am I just being unreasonable in expecting to be able to dump something like that?


    ;$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$;[-1]->();print

      Both my version and the current Data::Dumper() allow coderefs to be dumped. My module does it by default, Data::Dumper does it only on request.


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


        Both my version and the current Data::Dumper() allow coderefs to be dumped. My module does it by default, Data::Dumper does it only on request.

        I didn't ask about coderefs in general. I asked about closures specifically.


        ;$;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$;[-1]->();print