Re^3: directory tree to hash of arrays

Replies are listed 'Best First'.
Re^4: directory tree to hash of arrays by mabossert (Scribe) on Mar 08, 2014 at 04:23 UTC
I am fairly well familiar with how a hash and an array work...as well as a hash or arrays or vice-versa. I simply copied a snippet of your response. What I am looking for is a structure that will lend itself well to translation to JSON for display within a web application directory tree: `$VAR1 = { '/tmp/dir1' => { 'innerdir1' => [{'name' => 'file1', 'size' => 12345}, {'name' +=> 'file2', 'size' => 1223}] } };` [download]	[reply] [d/l]
Re^5: directory tree to hash of arrays by hippo (Archbishop) on Mar 08, 2014 at 10:29 UTC
That's a little inconsistent. Your outer dir is a hashref while your inner dir is an arrayref. Furthermore in some places (directories) you use the name as a key and in other places (files) as a value. I agree with kcott - it's always better to plan your data structure in detail first before considering the minutiae of how to construct it.	[reply]
Re^4: directory tree to hash of arrays by mabossert (Scribe) on Mar 14, 2014 at 02:18 UTC
Sorry if this appears as a double...I updated the original post...but wasn't sure if that would bubble up to you, so here is the same content Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get. First, the data structure I want: my @dk = ( {data => [ { text => 'data', data => { spriteCssClass => 'folder', 'fullPath' => '/data' }, contents => [ { text => 'raw', data => { spriteCssClass => 'folder', fullPath => '/data/raw' }, contents => [ { text => 'file1.csv', data => { spriteCssClass => 'csv +', fullPath => '/data/raw +/file1.csv', size => '12345' } }, { text => 'file2.csv', data => { spriteCssClass => 'csv +', fullPath => '/data/raw +/file2.csv', size => '123' } } ] }, { text => 'rdf', data => { spriteCssClass => 'folder', fullPath => '/data/rdf' }, contents => [ { text => 'file1.rdf', data => { spriteCssClass => 'rdf +', fullPath => '/data/raw +/file1.rdf', size => '123456' } }, { text => 'file2.csv', data => { spriteCssClass => 'rdf +', fullPath => '/data/raw +/file2.rdf', size => '1235' } } ] } ], } ] }); [download] Now, the code...which is as far as I have gotten, modifying the previously mentioned code and reference Here. sub _directoryTree { my ($directory,$filter) = @_; my (@files,@dirs); find(sub{ push @files, $File::Find::name; },$directory); foreach my $file(@files) { chomp $file; my $ref = \@dirs; my $truncated = $file; $truncated =~ s/^$filter//; foreach my $dir ( split /\//,$truncated ) { my $i = 0; next if $dir eq ''; $i++ while ( $ref->[$i] and $ref->[$i]{name} ne $dir ); my $r; if (-f $file) { my ($ext,$spriteclass); if ($file =~ /\.(\w+)$/) { $ext = $1; if ($ext eq 'html') { $spriteclass = 'html'; } elsif($ext =~ m/png\|jpg\|jpeg\|gif\|ico/) { $spriteclass = 'image'; } elsif($ext eq 'pdf') { $spriteclass = 'pdf'; } else { $spriteclass = 'html'; } } $r= $ref->[$i] \|\|= {'text' => $dir, 'data' => { 'spriteCssClass' => $spritecla +ss, 'fullPath' => $file, 'id' => $dir, 'size' => (-s $file), 'type' => $ext } } unless $dir =~ m/^\./; #, 'access +ed' => ctime(stat($file)->atime) } else { $r= $ref->[$i] \|\|= {'text' => $dir, 'files' => [], 'data' => { 'spriteCssClass' => 'folder', 'fullPath' => $file, 'id' => $dir, 'size' => 0, 'type' => 'folder' } }; } $ref = $r->{'files'}; } } return \@dirs; } [download]	[reply] [d/l] [select]
Re^5: directory tree to hash of arrays by kcott (Archbishop) on Mar 14, 2014 at 04:36 UTC
"Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get." Your "current code" is just a subroutine definition. You don't show how, or in what context, you're calling it. You don't say how it's behaving differently from your expections or whether it's generating error or warning messages. You don't indicate which part, or parts, you're specifically having difficulties with ("wrapping my head around"). I've already provided you with the link to "How do I post a question effectively?" in this thread: here it is again, please read it this time and follow its guidelines in any future postings. "First, the data structure I want:" So, you want an array with a single element which is a hashref; that hashref having a single key/value pair, the value being an arrayref; that arrayref having a single element which is a hashref; ... In some places you want a key called 'data' to have a value which is an arrayref and in other places you want a key with same name to have a value which is an hashref. Can you explain why you want a structure like that and how you propose to use it? I asked you to provide the data structure you'd like to generate from a fairly simple directory structure. You've refused to do that! Why? What you've posted is not related to any directory structure you've shown and is peppered with meaningless noise. For instance, am I supposed to know what "`spriteCssClass`" is? I'm not convinced that you know what data structure you actually want. This would be the reason why you're having difficulties writing code to generate it. About the only help I can provide you with at this time is to show a possible data structure. This is an example of what I was expecting from you. `{ name => 'dir_A', subs => [ { name => 'file_B', size => 99, }, { name => 'dir_C', subs => [ { name => 'file_D', size => 99, }, { name => 'dir_E', subs => [ { name => 'file_F', size => 99, }, ], }, { name => 'file_G', size => 99, }, ], }, { name => 'file_H', size => 99, }, ], }` [download] Once you get the structure right, you can add whatever other information you want (file types, permissions, modification times, etc.). But, you need to get the structure right first! -- Ken	[reply] [d/l] [select]
Re^6: directory tree to hash of arrays by mabossert (Scribe) on Mar 14, 2014 at 06:11 UTC
The subroutine takes a directory as an argument...then iterates over the directories and files found in that directory. The result needs to be sent as JSON to a web-application that uses the resulting data to populate a directory tree (file browser). The data structure you were poking at (array with single element which is a hashref...) is structured that way because the Javascript UI framework expects a JSON object with a "data" section that contains, not surprisingly, data...and a metadata section that contains other information about the datasource (e.g. if paging is used, how many records, time if retrieval, etc.). I didn't refuse to use your example...I simply thought that the actual structure I wanted was more useful. So, if it makes it easier, here is your structure: [ {information => { text => 'dir_A', data => { size => 99, type => 'folder' }, contents => [ { text => 'file_B', data => { size => 99, type => 'file' } }, { text => 'dir_C', data => { size => 99, type => 'folder' }, contents => [ { text => 'file_D', data => { size => 99, type => 'file' } }, { text => 'dir_E', data => { size => 99, type => 'folder' }, contents => [ { text => 'file_F', data => { size => 99, type => 'file' } } ] }, { text => 'file_G', data => { type => 'file', size => 99 } } ] }, { text => 'file_H', data => { type => 'file', size => 99 } } ] } } ] [download] The subroutine would be used thusly: `my $arrayref = _directoryTree('/usr/local/bin','/usr/local');` [download] The reason for the filter is that I don't want to expose the entire path to the user of the web-application. All they need to see is the directory structure starting at a certain level...so, rather that showing them a tree consisting of /home/user/somedirectory/web-app/public/data... The filter could be used to "start" with /public/data as the "root", rather than the whole thing. The arrayref is then passed to the web-app framework(Mojolicious) and rendered as JSON to the client side thusly: `get => 'directory_list' sub { my $self = shift; my $arrayref = _directoryTree('/usr/local/bin','/usr/local'); $self->render(json => $arrayref); };` [download] On a final note. I understand what you said about the two different uses of a key called "data". Since those are in two entirely different hashes, they have nothing to do with each other. The convention for the javascript framework is to construct a datasource with a "data" section containing the data and another data section for each record that contains any user-defined fields (e.g. those the framework does not specifically call out)...making it more sensible on the javascript side. If the field is in the documentation then just call object.text for instance, for an undocumented field, call object.data.customThing. Not my choice of conventions...I am just working with it.	[reply] [d/l] [select]

I am fairly well familiar with how a hash and an array work...as well as a hash or arrays or vice-versa. I simply copied a snippet of your response. What I am looking for is a structure that will lend itself well to translation to JSON for display within a web application directory tree:

$VAR1 = {
    '/tmp/dir1' => {
        'innerdir1' => [{'name' => 'file1', 'size' => 12345}, {'name' 
+=> 'file2', 'size' => 1223}]
    }
};
[download]

[reply]
[d/l]

That's a little inconsistent. Your outer dir is a hashref while your inner dir is an arrayref. Furthermore in some places (directories) you use the name as a key and in other places (files) as a value.

I agree with kcott - it's always better to plan your data structure in detail first before considering the minutiae of how to construct it.

[reply]

Sorry if this appears as a double...I updated the original post...but wasn't sure if that would bubble up to you, so here is the same content

Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get. First, the data structure I want:

my @dk = (  {data => [
                    {
                        text => 'data',
                        data => {
                            spriteCssClass => 'folder',
                            'fullPath' => '/data'
                        },
                        contents => [
                                  {
                                    text => 'raw',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/raw'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file1.csv',
                                                size => '12345'
                                            }
                                        },
                                        {
                                            text => 'file2.csv',
                                            data => {
                                                spriteCssClass => 'csv
+',
                                                fullPath => '/data/raw
+/file2.csv',
                                                size => '123'
                                            }
                                        }
                                        
                                    ]
                                  },
                                  {
                                    text => 'rdf',
                                    data => {
                                        spriteCssClass => 'folder',
                                        fullPath => '/data/rdf'
                                    },
                                    contents => [
                                        {
                                            text => 'file1.rdf',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file1.rdf',
                                                size => '123456'
                                            }
                                        },
                                        {
                                            text => 'file2.csv',
                                            data => {
                                                spriteCssClass => 'rdf
+',
                                                fullPath => '/data/raw
+/file2.rdf',
                                                size => '1235'
                                            }
                                        }
                                    ]
                                  }
                                  ],
                     }
                    ]
            });
[download]

Now, the code...which is as far as I have gotten, modifying the previously mentioned code and reference Here.

sub _directoryTree {
    my ($directory,$filter) = @_;
    my (@files,@dirs);
    
    find(sub{
        push @files, $File::Find::name;
    },$directory);
    foreach my $file(@files) {
        chomp $file;
        my $ref = \@dirs;
        my $truncated = $file;
        $truncated =~ s/^$filter//;
        
        foreach my $dir ( split /\//,$truncated ) {
            my $i = 0;
            next if $dir eq '';
            $i++ while ( $ref->[$i] and $ref->[$i]{name} ne $dir );
            my $r;
            if (-f $file) {
                my ($ext,$spriteclass);
                if ($file =~ /\.(\w+)$/) {
                    $ext = $1;
                    if ($ext eq 'html') {
                        $spriteclass = 'html';
                    }
                    elsif($ext =~ m/png|jpg|jpeg|gif|ico/) {
                        $spriteclass = 'image';
                    }
                    elsif($ext eq 'pdf') {
                        $spriteclass = 'pdf';
                    }
                    else {
                        $spriteclass = 'html';
                    }
                }
                
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'data' => {
                                        'spriteCssClass' => $spritecla
+ss,
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => (-s $file),
                                        'type'     => $ext
                                    }
                                    
                                   } unless $dir =~ m/^\./; #, 'access
+ed' => ctime(stat($file)->atime)
            }
            else {
                $r= $ref->[$i] ||= {'text' => $dir,
                                    'files' => [],
                                    'data' => {
                                        'spriteCssClass' => 'folder',
                                        'fullPath' => $file,
                                        'id'       => $dir,
                                        'size'     => 0,
                                        'type'     => 'folder'
                                    }
                                    
                                   };
            }
            $ref  = $r->{'files'};
        }
    }
    return \@dirs;
}
[download]

[reply]
[d/l]
[select]

"Here is the current code, which is not doing what I want it to, and am having trouble wrapping my head around how to get to the structure I want to get."

Your "current code" is just a subroutine definition. You don't show how, or in what context, you're calling it. You don't say how it's behaving differently from your expections or whether it's generating error or warning messages. You don't indicate which part, or parts, you're specifically having difficulties with ("wrapping my head around"). I've already provided you with the link to "How do I post a question effectively?" in this thread: here it is again, please read it this time and follow its guidelines in any future postings.

"First, the data structure I want:"

So, you want an array with a single element which is a hashref; that hashref having a single key/value pair, the value being an arrayref; that arrayref having a single element which is a hashref; ... In some places you want a key called 'data' to have a value which is an arrayref and in other places you want a key with same name to have a value which is an hashref. Can you explain why you want a structure like that and how you propose to use it?

I asked you to provide the data structure you'd like to generate from a fairly simple directory structure. You've refused to do that! Why?

What you've posted is not related to any directory structure you've shown and is peppered with meaningless noise. For instance, am I supposed to know what "spriteCssClass" is?

I'm not convinced that you know what data structure you actually want. This would be the reason why you're having difficulties writing code to generate it.

About the only help I can provide you with at this time is to show a possible data structure. This is an example of what I was expecting from you.

{
    name => 'dir_A',
    subs => [
        {
            name => 'file_B',
            size => 99,
        },
        {
            name => 'dir_C',
            subs => [
                {
                    name => 'file_D',
                    size => 99,
                },
                {
                    name => 'dir_E',
                    subs => [
                        {
                            name => 'file_F',
                            size => 99,
                        },
                    ],
                },
                {
                    name => 'file_G',
                    size => 99,
                },
            ],
        },
        {
            name => 'file_H',
            size => 99,
        },
    ],
}
[download]

Once you get the structure right, you can add whatever other information you want (file types, permissions, modification times, etc.). But, you need to get the structure right first!

-- Ken

[reply]
[d/l]
[select]

The subroutine takes a directory as an argument...then iterates over the directories and files found in that directory. The result needs to be sent as JSON to a web-application that uses the resulting data to populate a directory tree (file browser). The data structure you were poking at (array with single element which is a hashref...) is structured that way because the Javascript UI framework expects a JSON object with a "data" section that contains, not surprisingly, data...and a metadata section that contains other information about the datasource (e.g. if paging is used, how many records, time if retrieval, etc.).

I didn't refuse to use your example...I simply thought that the actual structure I wanted was more useful. So, if it makes it easier, here is your structure:

[
  {information => {
    text => 'dir_A',
     data => {
      size => 99,
      type => 'folder'
    },
    contents => [
      {
        text => 'file_B',
         data => {
           size => 99,
           type => 'file'
         }
      },
      {
        text => 'dir_C',
        data => {
           size => 99,
           type => 'folder'
         },
        contents => [
          {
            text => 'file_D',
            data => {
              size => 99,
              type => 'file'
            }
          },
          {
            text => 'dir_E',
            data => {
              size => 99,
              type => 'folder'
            },
            contents => [
              {
                text => 'file_F',
                data => {
                  size => 99,
                  type => 'file'
                }
              }
            ]
          },
          {
            text => 'file_G',
            data => {
              type => 'file',
              size => 99
            }
          }
        ]
      },
      {
        text => 'file_H',
        data => {
          type => 'file',
          size => 99
        }
      }
    ]
  }
  }
]
[download]

The subroutine would be used thusly:

my $arrayref = _directoryTree('/usr/local/bin','/usr/local');
[download]

The reason for the filter is that I don't want to expose the entire path to the user of the web-application. All they need to see is the directory structure starting at a certain level...so, rather that showing them a tree consisting of /home/user/somedirectory/web-app/public/data... The filter could be used to "start" with /public/data as the "root", rather than the whole thing. The arrayref is then passed to the web-app framework(Mojolicious) and rendered as JSON to the client side thusly:

get => 'directory_list' sub {
  my $self = shift;
  my $arrayref = _directoryTree('/usr/local/bin','/usr/local');
  $self->render(json => $arrayref);
};
[download]

On a final note. I understand what you said about the two different uses of a key called "data". Since those are in two entirely different hashes, they have nothing to do with each other. The convention for the javascript framework is to construct a datasource with a "data" section containing the data and another data section for each record that contains any user-defined fields (e.g. those the framework does not specifically call out)...making it more sensible on the javascript side. If the field is in the documentation then just call object.text for instance, for an undocumented field, call object.data.customThing. Not my choice of conventions...I am just working with it.

[reply]
[d/l]
[select]