in reply to directory tree to hash of arrays

G'day mabossert,

It would have been a lot more helpful to show us your code and explain what parts you were having difficulty with.

You say you found a solution but had a problem understanding the code. If you told us what that problem was, we could help you with it.

The actual data structure you want to create is completely unclear. You say "What I want to do is have something like this ..." but what you show doesn't seem to equate with a tree structure indicated in the title ("directory tree to hash of arrays"). Perhaps look in "perldsc - Perl Data Structures Cookbook" and try to determine what you really want.

As it happens, just yesterday I wrote "Re: Recursive Directory print". I've made some minor modifications: this may do what you want. 

#!/usr/bin/env perl

use strict;
use warnings;
use autodie;

my %dir_tree;
my $root_dir = '.';

build_dir_tree(\%dir_tree, $root_dir);

sub build_dir_tree {
    my ($tree, $path) = @_;

    opendir(my $dh, $path);

    for (readdir $dh) {
        next if /^(?:\.|\.\.)$/;

        if (-d "$path/$_") {
            $tree->{$_} = {};
            build_dir_tree($tree->{$_}, "$path/$_");
        }
        else {
            $tree->{$_} = -s "$path/$_";
        }
    }
}

{
    use Data::Dumper;
    local $Data::Dumper::Indent = 1;
    local $Data::Dumper::Sortkeys = 1;
    print Dumper \%dir_tree;
}

[download]

The output is almost 3,000 lines long so I won't post it here. Here's an extract to give you an idea of the structure it produces. 

  ...
  'pm_multi_line_match.pl' => 1025,
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      'Build.PL' => 651,
      'Changes' => 92,
      'MANIFEST' => 136,
      'Makefile.PL' => 730,
      'README' => 1316,
      'ignore.txt' => 120,
      'lib' => {
        'My' => {
          'Example.pm' => 5155
        }
      },
      't' => {
        '00.load.t' => 117,
        'perlcritic.t' => 454,
        'pod-coverage.t' => 437,
        'pod.t' => 405
      }
    }
  },
  'pm_multiline_parse.pl' => 835,
  'pm_multiline_parse_hdb_prob.pl' => 601,
  ...

[download]

If that's what you want, then fine — just use it. If you have follow-up questions, please address my opening remarks and adhere to the guidelines in "How do I post a question effectively?".

And, in case you didn't know, the JSON module will do the conversion.

-- Ken

Replies are listed 'Best First'.
Re^2: directory tree to hash of arrays
by mabossert (Scribe) on Mar 08, 2014 at 01:28 UTC

    Thanks...I guess I thought it was clearer than it actually was. What I wanted was similar to what you provided...but instead of each key being the file name, I want to be able to have a series of hashes in an array...such that it would be populated like:

    The place I get lost in the code at the bottom is that the hash is returned...but I don't see how to take that and push it onto an array with the way the code was written

    {name => 'pm_multi_line_match.pl', size => 1025},
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      {name => 'Build.PL', size => 651},
...etc

    [download]
    #!/usr/bin/env perl

use strict;
use warnings;
use File::Find;
use Data::Dumper;

my $rootDir = '/tmp/somedir';
$Data::Dumper::Indent = 1;

build_tree(my $tree, $rootDir);
print Dumper($tree);



sub build_tree {
  my $node = $_[0] = {};
  my @s;
  find( sub {
    push @$node, (pop @s)->[1] while @s and $File::Find::dir ne $s[-1]
+[0];
    $node->{'name'} = $_ if -f;
    $node->{'size'} = -s if -f;
    # Here is where I am lost...How do I get this pushed into an array
+?
    return $node if -f;
    push @s, [ $File::Find::name, $node ];
    $node = $node->{$_} = {};
  }, $_[1]);
  $_[0]{$_[1]} = delete $_[0]{'.'};
}

    [download]
[reply]
[d/l]
[select]
      "What I wanted was similar to what you provided...but instead of each key being the file name, I want to be able to have a series of hashes in an array...such that it would be populated like:"

      You've forgotten to post the part that tells us what "it would be populated like"!

      If it was supposed to be this (which appears after the next paragraph): 

      {name => 'pm_multi_line_match.pl', size => 1025},
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      {name => 'Build.PL', size => 651},
...etc

      [download]

      then I'm left convinced that you don't really have a clear picture of the data structure you want.

      • What you've posted there has no container: what structure holds that data?
      • It starts with a hashref: was there supposed to be a key before it?
      • The next line has a key ('pm_multi_mod_distro_test') whose value is a hashref.
      • That hashref has a key ('My-Example') whose value is a hashref.
      • That hashref has no keys. You show another hashref where the key should be.
      • Basically, this is not a Perl data structure!

      I suggest the first thing you do is follow the link I already gave you ("perldsc - Perl Data Structures Cookbook") and learn about Perl data structures.

      When you've done that, tell us exactly, and in full (between <code>...</code> tags), what Perl data structure you'd like to generate from this directory structure (assuming all files have a size of 99): 

      dir_A
+-file_B
+-dir_C
  +-file_D
  +-dir_E
    +-file_F
  +-file_G
+-file_H

      [download]

      When we know what your goal is, we can help you to achieve it.

      -- Ken

[reply]
[d/l]
[select]

        I am fairly well familiar with how a hash and an array work...as well as a hash or arrays or vice-versa. I simply copied a snippet of your response. What I am looking for is a structure that will lend itself well to translation to JSON for display within a web application directory tree:

        $VAR1 = {
    '/tmp/dir1' => {
        'innerdir1' => [{'name' => 'file1', 'size' => 12345}, {'name' 
+=> 'file2', 'size' => 1223}]
    }
};

        [download]
[reply]
[d/l]

        Sorry if this appears as a double...I updated the original post...but wasn't sure if that would bubble up to you, so here is the same content

        Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get. First, the data structure I want:

        my @dk = (  {data => [
                    {
                        text => 'data',
                        data => {
                            spriteCssClass => 'folder',
                            'fullPath' => '/data'
                        },
                        contents => [
                                  {
                                    text => 'raw',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/raw'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file1.csv',
                                                size => '12345'
                                            }
                                        },
                                        {
                                            text => 'file2.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file2.csv',
                                                size => '123'
                                            }
                                        }
                                        
                                    ]
                                  },
                                  {
                                    text => 'rdf',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/rdf'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.rdf',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file1.rdf',
                                                size => '123456'
                                            }
                                        },
                                        {
                                            text => 'file2.csv',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file2.rdf',
                                                size => '1235'
                                            }
                                        }
                                    ]
                                  }
                                  ],
                     }
                    ]
            });

        [download]

        Now, the code...which is as far as I have gotten, modifying the previously mentioned code and reference Here.

        sub _directoryTree {
    my ($directory,$filter) = @_;
    my (@files,@dirs);
    
    find(sub{
        push @files, $File::Find::name;
    },$directory);
    foreach my $file(@files) {
        chomp $file;
        my $ref = \@dirs;
        my $truncated = $file;
        $truncated =~ s/^$filter//;
        
        foreach my $dir ( split /\//,$truncated ) {
            my $i = 0;
            next if $dir eq '';
            $i++ while ( $ref->[$i] and $ref->[$i]{name} ne $dir );
            my $r;
            if (-f $file) {
                my ($ext,$spriteclass);
                if ($file =~ /\.(\w+)$/) {
                    $ext = $1;
                    if ($ext eq 'html') {
                        $spriteclass = 'html';
                    }
                    elsif($ext =~ m/png|jpg|jpeg|gif|ico/) {
                        $spriteclass = 'image';
                    }
                    elsif($ext eq 'pdf') {
                        $spriteclass = 'pdf';
                    }
                    else {
                        $spriteclass = 'html';
                    }
                }
                
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'data' => {
                                        'spriteCssClass' => $spritecla
+ss,
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => (-s $file),
                                        'type'     => $ext
                                    }
                                    
                                   } unless $dir =~ m/^\./; #, 'access
+ed' => ctime(stat($file)->atime)
            }
            else {
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'files' => [],
                                    'data' => {
                                        'spriteCssClass' => 'folder',
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => 0,
                                        'type'     => 'folder'
                                    }
                                    
                                   };
            }
            $ref  = $r->{'files'};
        }
    }
    return \@dirs;
}

        [download]
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom