mabossert has asked for the wisdom of the Perl Monks concerning the following question:

Ladies and/or gents, I have found a previous solution that gets really close to what I need, but I guess I am just misunderstanding how it actually works. What I want is to be able to convert a directory structure to a hash or arrays such that it will be easily converted to JSON for presentation to a web application. Here.

UPDATE: I didn't want to delete the original post...so, here is where I am:

Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get. First, the data structure I want:

my @dk = (  {data => [
                    {
                        text => 'data',
                        data => {
                            spriteCssClass => 'folder',
                            'fullPath' => '/data'
                        },
                        contents => [
                                  {
                                    text => 'raw',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/raw'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file1.csv',
                                                size => '12345'
                                            }
                                        },
                                        {
                                            text => 'file2.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file2.csv',
                                                size => '123'
                                            }
                                        }
                                        
                                    ]
                                  },
                                  {
                                    text => 'rdf',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/rdf'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.rdf',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file1.rdf',
                                                size => '123456'
                                            }
                                        },
                                        {
                                            text => 'file2.rdf',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file2.rdf',
                                                size => '1235'
                                            }
                                        }
                                    ]
                                  }
                                  ],
                     }
                    ]
            });

[download]

Now, the code...which is as far as I have gotten, modifying the previously mentioned code and reference Here.

sub _directoryTree {
    my ($directory,$filter) = @_;
    my (@files,@dirs);
    
    find(sub{
        push @files, $File::Find::name;
    },$directory);
    foreach my $file(@files) {
        chomp $file;
        my $ref = \@dirs;
        my $truncated = $file;
        $truncated =~ s/^$filter//;
        
        foreach my $dir ( split /\//,$truncated ) {
            my $i = 0;
            next if $dir eq '';
            $i++ while ( $ref->[$i] and $ref->[$i]{name} ne $dir );
            my $r;
            if (-f $file) {
                my ($ext,$spriteclass);
                if ($file =~ /\.(\w+)$/) {
                    $ext = $1;
                    if ($ext eq 'html') {
                        $spriteclass = 'html';
                    }
                    elsif($ext =~ m/png|jpg|jpeg|gif|ico/) {
                        $spriteclass = 'image';
                    }
                    elsif($ext eq 'pdf') {
                        $spriteclass = 'pdf';
                    }
                    else {
                        $spriteclass = 'html';
                    }
                }
                
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'data' => {
                                        'spriteCssClass' => $spritecla
+ss,
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => (-s $file),
                                        'type'     => $ext
                                    }
                                    
                                   } unless $dir =~ m/^\./; #, 'access
+ed' => ctime(stat($file)->atime)
            }
            else {
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'files' => [],
                                    'data' => {
                                        'spriteCssClass' => 'folder',
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => 0,
                                        'type'     => 'folder'
                                    }
                                    
                                   };
            }
            $ref  = $r->{'files'};
        }
    }
    return \@dirs;
}

[download]

Replies are listed 'Best First'.
Re: directory tree to hash of arrays
by Kenosis (Priest) on Mar 08, 2014 at 00:27 UTC

    Perhaps the following will be helpful:

    use strict;
use warnings;
use Cwd;
use JSON;

my %hash;
my $currDir = Cwd::cwd();

push @{ $hash{$currDir} }, { name => $_, size => ( stat($_) )[7] } for
+ <*>;
my $j = encode_json \%hash;
print $j;

    [download]

    Sample output:

    {"/home/user/programming/Perl/tmp":[{"name":"File1.txt","size":63},{"n
+ame":"File2.txt","size":55},{"name":"temp5.pl","size":196},{"name":"t
+est.pl","size":197}]}

    [download]

    When run, it gets the current dir, uses it as the hash key whose value is a reference to an array of (file and filesize) hashes. That hash is later json encoded. Substitute your own path in $currDir, as needed.

[reply]
[d/l]
[select]

      Thanks for the help! I should have been more clear in the original post. I need it to be recursive...Hence the link I put in the original post.

[reply]
Re: directory tree to hash of arrays
by kcott (Archbishop) on Mar 08, 2014 at 01:09 UTC

    G'day mabossert,

    It would have been a lot more helpful to show us your code and explain what parts you were having difficulty with.

    You say you found a solution but had a problem understanding the code. If you told us what that problem was, we could help you with it.

    The actual data structure you want to create is completely unclear. You say "What I want to do is have something like this ..." but what you show doesn't seem to equate with a tree structure indicated in the title ("directory tree to hash of arrays"). Perhaps look in "perldsc - Perl Data Structures Cookbook" and try to determine what you really want.

    As it happens, just yesterday I wrote "Re: Recursive Directory print". I've made some minor modifications: this may do what you want. 

    #!/usr/bin/env perl

use strict;
use warnings;
use autodie;

my %dir_tree;
my $root_dir = '.';

build_dir_tree(\%dir_tree, $root_dir);

sub build_dir_tree {
    my ($tree, $path) = @_;

    opendir(my $dh, $path);

    for (readdir $dh) {
        next if /^(?:\.|\.\.)$/;

        if (-d "$path/$_") {
            $tree->{$_} = {};
            build_dir_tree($tree->{$_}, "$path/$_");
        }
        else {
            $tree->{$_} = -s "$path/$_";
        }
    }
}

{
    use Data::Dumper;
    local $Data::Dumper::Indent = 1;
    local $Data::Dumper::Sortkeys = 1;
    print Dumper \%dir_tree;
}

    [download]

    The output is almost 3,000 lines long so I won't post it here. Here's an extract to give you an idea of the structure it produces. 

      ...
  'pm_multi_line_match.pl' => 1025,
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      'Build.PL' => 651,
      'Changes' => 92,
      'MANIFEST' => 136,
      'Makefile.PL' => 730,
      'README' => 1316,
      'ignore.txt' => 120,
      'lib' => {
        'My' => {
          'Example.pm' => 5155
        }
      },
      't' => {
        '00.load.t' => 117,
        'perlcritic.t' => 454,
        'pod-coverage.t' => 437,
        'pod.t' => 405
      }
    }
  },
  'pm_multiline_parse.pl' => 835,
  'pm_multiline_parse_hdb_prob.pl' => 601,
  ...

    [download]

    If that's what you want, then fine — just use it. If you have follow-up questions, please address my opening remarks and adhere to the guidelines in "How do I post a question effectively?".

    And, in case you didn't know, the JSON module will do the conversion.

    -- Ken

[reply]
[d/l]
[select]

      Thanks...I guess I thought it was clearer than it actually was. What I wanted was similar to what you provided...but instead of each key being the file name, I want to be able to have a series of hashes in an array...such that it would be populated like:

      The place I get lost in the code at the bottom is that the hash is returned...but I don't see how to take that and push it onto an array with the way the code was written

      {name => 'pm_multi_line_match.pl', size => 1025},
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      {name => 'Build.PL', size => 651},
...etc

      [download]
      #!/usr/bin/env perl

use strict;
use warnings;
use File::Find;
use Data::Dumper;

my $rootDir = '/tmp/somedir';
$Data::Dumper::Indent = 1;

build_tree(my $tree, $rootDir);
print Dumper($tree);



sub build_tree {
  my $node = $_[0] = {};
  my @s;
  find( sub {
    push @$node, (pop @s)->[1] while @s and $File::Find::dir ne $s[-1]
+[0];
    $node->{'name'} = $_ if -f;
    $node->{'size'} = -s if -f;
    # Here is where I am lost...How do I get this pushed into an array
+?
    return $node if -f;
    push @s, [ $File::Find::name, $node ];
    $node = $node->{$_} = {};
  }, $_[1]);
  $_[0]{$_[1]} = delete $_[0]{'.'};
}

      [download]
[reply]
[d/l]
[select]
        "What I wanted was similar to what you provided...but instead of each key being the file name, I want to be able to have a series of hashes in an array...such that it would be populated like:"

        You've forgotten to post the part that tells us what "it would be populated like"!

        If it was supposed to be this (which appears after the next paragraph): 

        {name => 'pm_multi_line_match.pl', size => 1025},
  'pm_multi_mod_distro_test' => {
    'My-Example' => {
      {name => 'Build.PL', size => 651},
...etc

        [download]

        then I'm left convinced that you don't really have a clear picture of the data structure you want.

        • What you've posted there has no container: what structure holds that data?
        • It starts with a hashref: was there supposed to be a key before it?
        • The next line has a key ('pm_multi_mod_distro_test') whose value is a hashref.
        • That hashref has a key ('My-Example') whose value is a hashref.
        • That hashref has no keys. You show another hashref where the key should be.
        • Basically, this is not a Perl data structure!

        I suggest the first thing you do is follow the link I already gave you ("perldsc - Perl Data Structures Cookbook") and learn about Perl data structures.

        When you've done that, tell us exactly, and in full (between <code>...</code> tags), what Perl data structure you'd like to generate from this directory structure (assuming all files have a size of 99): 

        dir_A
+-file_B
+-dir_C
  +-file_D
  +-dir_E
    +-file_F
  +-file_G
+-file_H

        [download]

        When we know what your goal is, we can help you to achieve it.

        -- Ken

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom