FWIW: I find the output of Data::Dump far more readable that yaml, and it doesn't make the Pythonesque mistake of relying upon invisible characters for structural integrity.

I did find the standard maximum width of 60 rather limiting, so I patched my version to make it a global so that I can adjust it.

Of course, if you're squeamish about using eval it may not be what you want, but I see no greater risk in the contents of local data files, than local source files. And I see no reason to trade Perl's well-proven, reasonably efficient parser, for less well-proven, invariably highly inefficient, implementations of a design-by-committee spec. in order to avoid using eval.

Data::Dump suffers from the same problem as YAML::Syck, that the huge block of text data gets turned into a long, unruly string. And while I do use Data::Dump a lot for debugging, when dealing with deeply-nested objects I find it gets way too wide to read way too quickly, while YAML is usually much easier to follow, due to the amount of whitespace that ends up at the beginning of the line... i.e.

### An object like this...
my $obj = bless( {
    this_object_has_some_long_keys => {
        and_some_of_them_are_nested => {
            fairly_deeply => 'Foo!',
        }
    },
}, 'Some::Random::ClassName' );

### Will end up like this with Data::Dumper...
bless({
  "this_object_has_some_long_keys" => { "and_some_of_them_are_nested" 
+=> { fairly_deeply => "Foo!" } },
}, "Some::Random::ClassName")

### YAML output is more readable, IMHO
--- !!perl/hash:Some::Random::ClassName
this_object_has_some_long_keys:
  and_some_of_them_are_nested:
    fairly_deeply: Foo!
[download]

Ultimately, though, your comment did lead me to a solution which works. This is what I'm using now...

From the serializer...

$fh->print(
    "package MyApp::TestData::$digest;\n",
    "use parent 'Class::Data::Accessor';\n\n\n",
    "use YAML qw( Load );\n",
    'sub data($$) { __PACKAGE__->mk_classaccessor( @_ ) }'."\n\n",
    "data filename => ".dump( $self->filename ).";\n\n",
    "data store => Load( <<'$tag' );\n",
        YAML::Dump( $store ),
    "$tag\n\n",
    "data content => <<'$tag';\n",
        $self->content,
    "$tag\n",
    "return __PACKAGE__;\n",
);
[download]

Which produces an output file that looks like this:

package MyApp::TestData::PUHmx80zfPHoDloIOrexGA;
use parent 'Class::Data::Accessor';
use YAML qw( Load );
sub data($$) { __PACKAGE__->mk_classaccessor( @_ ) }

data filename => "test-data/some-filename";

data store => Load( <<'___END_OF_TEST_DATA_SECTION___' );
--- &1 !!perl/hash:MyApp::Store
children:
  - &2 !!perl/hash:MyApp::Store
___END_OF_TEST_DATA_SECTION___

data content => <<'___END_OF_TEST_DATA_SECTION___';
Original file contents here...
___END_OF_TEST_DATA_SECTION___
return __PACKAGE__;
[download]

Then, my regression test scripts can just do this:

for my $file ( <*.pl> ) {
    ok( my $class = do $file, "Loaded $file" );
    my $p = MyApp::Processor->new(
        filename    => $class->filename,
        content     => $class->content,
    );
    eq_or_diff( $p->store, $class->store );
}
[download]

---

www.jasonkohles.com
We're not surrounded, we're in a target-rich environment!

Why do you need a human readable serialisation format when your data looks perfectly human readable currently?

The problem appears simple. You have multiple data formats. You want to compare data. To do this you need a series of filters to standardise the data into a standard format. You need to validate your conversions.

The desire to use a human readable format suggest you "human" plan to read the data and decide if it is valid. Why do that after every change? Why not leverage modules, /t test.t files and make test to do this?

The point of making it human-readable wasn't that it was going to be reviewed by a human at test time (in fact I am using test modules just as you suggest) it was simply to make it easier for me to review the original data in the event that the test fails, so I can more easily determine what went wrong.

True, the data is currently perfectly readable, but I would like to store it in the same file as the processed data that is associated with it to make it easier to manage by not having the data scattered across multiple files.

In essence, what I'm trying to do is something like this...

my $data = file( shift() )->slurp;

### At dev-time, when this processor is known to be working

my $tst1 = MyApp::Processor->process( $data );
YAML::DumpFile( $file, $data, $test1 );

### Then, sometime later in a test script...

my @files = <test-files/*.yml>;
plan tests => scalar @files;

for my $file ( @tests ) {
    my ( $data, $test1 ) = YAML::LoadFile( $file );
    my $test2 = MyApp::Processor->process( $data );

    eq_or_diff( $test1, $test2, "$file not broken yet!" );
}
[download]

---

www.jasonkohles.com
We're not surrounded, we're in a target-rich environment!

Nothing you have said so far explains the need for YAML/Data:Dump etc serialisation to me. You data is already human readable. Storing the original and processed data in the same file is a simple as adding a separator. You don't need any fancy modules to do it.

while(<DATA>) {
    if (m/<ORIG DATA ABOVE MUNGE BELOW>/) {
        $munge .= $_ while <DATA>;  # slurp
    } else {
        $orig .= $_;
    }
}
[download]

As an added benefit of keeping it simple you can leverage diff to do the data comparison to do your eq_or_diff() routine if a simple eq test fails. I really think you are over complicating the task by adding a middleware serialisation layer. You are not actually using it to reconstitute a data structure, nor is there any real need as all you want to do is reformat the old data into the new format so you can process it. Why add useless middleware that only offers the opportunity to include bugs for no real gain?

As I see it you need a base class that has the functions:

my ($orig,$munge) = load_file($file);  # munge may be NULL
my $data          = parse($orig);      # process current format data o
+nly
my $cur_format    = serialise($data);  # output current format
write_file($orig, $current_format);    # write to file with separator
my $invalid       = eq_or_diff($munge,$cur_format); 
print "$file\n$invalid\n" if $invalid; # diff output, null if OK
[download]

Each filter class only requires a parse() method to generate whatever data structure you want to work with in your ultimate program.

You probably already have parse code to work with current data. The serialise method simply writes this data struct back into a sting that you can save. For current data this may or may not be identical to the current data format, but the process is valid if a base class parse on the original and munge data serialises to the same result as it is then round tripping.

Essentially what I am saying is don't use serialisation middleware. Write your own code that takes your data structure (which you need) and serialises it *into the current format* (which you need, mostly for validation). The filters become simply a parse method to generate your standard internal data structure. Note that if your internal data structure uses hashes ensure you apply a sort or a list ordering to the keys during serialisation. If you don't it will probably bite you. It has bitten me before as key return order is not guaranteed and is different in different versions of perl on different OSs for exactly the same data.

Doing it this way gives you:

1. Human readable output
2. The old data in new data format in the same file
3. A format that can easily be munged by diff to show the exact differences, probably in the most intuitively understandable format.
4. No useless middleware bugs to deal with. You will personally own all bugs :-)
5. A simple one method filter that does the absolute minimal task required - convert old data into a standardised internal representation ready to either work with or write back to file.

JSON uses the same subset of data structures as YAML::Tiny.

If he wants a "human-readable" storage format that supports objects, JSON won't help.

So is Data::Dumper, especially for perl programmers :p

I like JSON myself and use it extensively for data transfer purposes (it's about as compact as plaintext serialization format could ever be), but it's not particularly human-readable, even when pretty-printed. This is especially the case with multi-line strings mentioned in this example.

Still, it might be enough for debugging purposes. But so is any other text serialization format. Even (G-d forbid!) XML. With CDATA sections.

JSON can't round-trip perl objects without extra work, no matter which module you use to produce it.

I seem to have missed YAML::XS when looking at other YAML implementations though, and it passes all the failing test scripts I sent in with my YAML bug reports, so I'll probably start using that for all my YAML needs now.

---

www.jasonkohles.com
We're not surrounded, we're in a target-rich environment!

Unless there is an option I missed somewhere, XML::Simple can't roundtrip perl objects either...

use Test::More tests => 1;
use Test::Differences;
use XML::Simple;

my $in = bless( { foo => 'bar' }, 'Something' );
my $out = XMLin( XMLout( $in ) );
eq_or_diff( $in, $out );
__END__
1..1
not ok 1
#   Failed test at foo.pl line 8.
# +----+------------------+----------------+
# | Elt|Got               |Expected        |
# +----+------------------+----------------+
# *   0|bless( {          |{               *
# |   1|  foo => 'bar'    |  foo => 'bar'  |
# *   2|}, 'Something' )  |}               *
# +----+------------------+----------------+
# Looks like you failed 1 test of 1.
[download]

---

www.jasonkohles.com
We're not surrounded, we're in a target-rich environment!

Sorry, I missed that you wanted to handle blessed objects. I think you'd better stick with something that understands perl then, like Data::Dumper.

(Got, Expected) => (Expected, Got)
[download]

Please note that the key return order in a hash is not guaranteed by perl. While it will be constant for a given version of perl on one OS it may well change with a different version of perl or even the same version on a different OS. This has bitten me before when writing tests for CPAN modules. If you are stringifying hashes for comparison purposes make sure you apply a sort to the keys first.