jimpudar has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks,

I am looking for a function that can take a string produced by Data::TreeDumper and return a Perl hash.

Does anyone know if this already exists? If not, I will take a shot and upload it to CPAN.

To give some background on why I want this, I need to parse a log file produced by a Perl application (which I can not change), but some log messages take up multiple lines because they include a hash printed with TreeDumper.

As a result, quickly grepping for some information is very difficult. I want to write a Perl script that parses the log file and outputs one log message per line. I will probably use JSON to re encode the hash, but I would also be open to other suggestions.

If it matters, the logger has set the option $Data::TreeDumper::Displayaddress = 0;

Best,

Jim

Replies are listed 'Best First'.
Re: TreeDumper Parser / Inverse?
by Anonymous Monk on Aug 09, 2017 at 14:19 UTC
    Some quick testing reveals that it's not always possible to reconstruct the exact input unambiguously. Because of this, the usefulness of a CPAN module is questionable. You might be able to put together something that works well enough with the data you have, though.
    use Data::TreeDumper; $Data::TreeDumper::Displayaddress = 0; print DumpTree({ a => "z\n", b => "z[\\n]", 'c = d' => 'e = f', }); __END__ |- a = z[\n] |- b = z[\n] `- c = d = e = f
      For the record, here is the code I am using which works well enough for my purposes. Thanks again to everyone for the responses.
      sub untree { my $lines = shift; my %obj; my @stack; foreach (@$lines) { /([|`][-])/; my $key_level = $-[0] / 3; if ( $key_level < @stack ) { pop @stack until $key_level == @stack; } my $current = \%obj; $current = $current->{$_} foreach @stack; if (/(\w+)\s=\s(\w*)\s*$/) { $current->{$1} = $2 eq 'undef' ? undef : $2; } elsif (/(\w+)\s\(no\selements\)/) { $current->{$1} = []; } elsif (/(\w+)\s=\s*$/) { $current->{$1} = ''; } elsif (/(\w+)\s*$/) { push @stack, $1; $current->{$1} = {}; } } return \%obj; }
      Ah, very good point. This type of data I had not considered. Thanks!
Re: TreeDumper Parser / Inverse?
by thanos1983 (Parson) on Aug 09, 2017 at 07:59 UTC

    Hello jimpudar,

    Is it possible to provide us with sample of input data and expected output. For me is a bit confusing your questions.

    Maybe there is no module to do what you want but we could try to create a solution for you by playing around with the data.

    Looking forward to your update, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!

      Hi thanos,

      To clarify, the structures I am trying to parse are generated by TreeDumper with $Data::TreeDumper::Displayaddress = 0.

      Here is a quick and dirty function I have whipped up which suffices for my possible inputs, but is not quite correct for all possible inputs. Specifically, it will interpret arrays as hashes with keys 0 .. n.

      #!/usr/bin/env perl use strict; use warnings FATAL => 'all'; use feature qw(say); use Test::Deep qw(eq_deeply); sub parse_treedump { my $lines = shift; my @lines = @$lines; my %obj; my @stack; my $parse_next_line = sub { my $line = shift @lines; my $key_indicator = $line =~ /\|-/ ? '|-' : '`-'; my $key_idx = index( $line, $key_indicator ); my $level = @stack; my $key_level = $key_idx / 3; if ($key_level < $level) { pop @stack until $key_level == ($level = @stack); } my $current_obj = \%obj; $current_obj = $current_obj->{$_} foreach @stack; if ($line =~ /(\w+)\s=\s(.*)$/ && $key_level == $level) { $current_obj->{$1} = $2 eq 'undef' ? undef : $2; } elsif ($line =~ /(\w+)\s\(no\selements\)/) { $current_obj->{$1} = [ ]; } elsif ($line =~ /(\w+)\s=$/) { $current_obj->{$1} = ''; } elsif ($line =~ /(\w+)/ && $key_level == $level) { push @stack, $1; $current_obj->{$1} = { }; } }; $parse_next_line->() while @lines; return \%obj } my $expected_hash = { one => 'two', three => { four => 'five', six => 'seven', eight => { nine => 'ten' }, eleven => [ ], twelve => undef, }, thirteen => 'fourteen' }; my $lines = []; while (<DATA>) { chomp; push @$lines, $_; } close DATA; my $output_hash = parse_treedump($lines); if (eq_deeply($output_hash, $expected_hash)) { say "OK"; } 1; __DATA__ |- one = two |- three | |- four = five | |- six = seven | |- eight | | `- nine = ten | |- eleven (no elements) | `- twelve = undef `- thirteen = fourteen

      As you can see, the TreeDump output in the __DATA__ section gets converted to the expected hash. I would prefer it to work for all possible inputs...

      Best,
      Jim

        Here's a crazy idea. Apply some simple transforms to the the TreeDump output:
        '|- ' => '', '| ' => '', '`-' => '', '=' => ':', 'undef' => 'null',
        You may need some more, but after that the text will be in YAML format, so you can use one of the YAML parsers to convert it to a hash.
        my $parse_next_line = sub { my $line = shift @lines; ... }; $parse_next_line->() while @lines;
        This is very silly. Just write
        foreach my $line (@lines) { ... }
        I kind of like this solution, but YMMV. The approach isn't much different from yours.
        my @stack = ({}); while (<DATA>) { s/\s+$//; # remove trailing space and newline my $depth = 0; $depth++ while $depth < @stack && s/^[|` ][- ] //; die 'malformed input' if $depth < 1; splice @stack, $depth; my $obj = $stack[-1]; if (/^(.*?) = ?(.*)/) { $obj->{$1} = $2 eq 'undef' ? undef : $2; } elsif (s/ \(no elements\)$//) { $obj->{$_} = []; } else { push @stack, $obj->{$_} = {}; } } my $output_hash = $stack[0];