Hi!
You are probably right, it becomes an X-Y problem. I will try to do my best to explain the idea of what I'm trying to do and what lead me to opening the current question. As I explained in previous topic (https://www.perlmonks.org/?node_id=11130389), I'm trying to create a bash script "on the fly" output an array of paths that does the following three stages:
1. Create the same directory hierarchy.
2. Copy the files.
3. Create the same links.
For that I can do:
1. I can use mkdir -p to create the full hierarchy based on the path.
2. I can use scp/rsync for copying (as it's inside container).
3. I can use ln -s to create the links.
So I wanted to build a structure that will contain all the information (links, directories, files). I came up with the following structure:
{ "/": { "type": "dir", "files": [ { "usr": { "type": "dir", "files": [ { "vsa": { "type": "link-dir", "source": "/root/site/tools/gauv" } } ] } }, { "root": { "type": "dir", "files": [ { "site": { "type": "dir", "files": [ { "tools": { "type": "dir", "files": [ { "gauv": { "type": "dir", "files": [ { "pkgs": { "type": "dir", "files": [ { "python3": { "type": "dir", "files": [ { "3.6.3a": { "type": "dir-link", "source": "/usr/vsa/pkgs +/python3/3.6.3" } }, { "3.6.3": { "type": "dir", "links": [ { "lib": { "type": "dir", "links": [] } }, { "bin": { "type": "dir", "links": [] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } }
Which contains only the path /usr/vsa/pkgs/python3/3.6.3a/bin/python3.6 with it's links (as I described in the question). So I will parse each path and create this structure. Once I have this structure, I can extract all of the directories, files, and links (dir links and file links) into arrays and use them to build the bash script (write bash commands based on those paths into a file). That's the purpose of that whole idea.
So my strategy was:
1. Parse each path (by getting all subpaths and links) and insert into an array on all path.
2. Check the type of each path in the array (link, file, directory) and insert into the structure.
3. Extract arrays of dir paths, array of dir links, array of files, array of file links.
4. Iterate over each array and create the bashe script.

I'm having trouble with steps 1-2. In the current node I ask about step 1. I wanted to parse each path and split it into sub paths. Then check each subpath if it's a link and if so, I will insert the target of the link into the array, change all of the other subpaths (for example if I have (/a,/a/b/,/a/b/d) and /a/b->/e/f then it should be (/a,/a/b/,/e/f,/e/f/d)). I should also handle two special cases here:
1. Relative links - not sure how to handle with it currently. I have tried to handle with only local relative links like /a/b/c -> /a/b/d but it's getting complicated.
2. Recursive links - for example /a/b -> /c/d -> e/f ...). That's why I used while(1).


To sum up, those are the big questions:
1. What would be the best design strategy to implement here? Was my idea good?
2. If so, Is my suggested structure good enough? How would you change it?


Now, for what you suggested:
1. The idea is to create a bash script that copies the environment into a container. rsync can help me here but I it can come in handy in step 4 (while copying files, instead of scp). I can't use rsync on the whole directory because it will then copy files that are not in the array of paths. Assume you have in the input array of path (/a/b/1.file, /a/b/2.file) and you also have 3.file inside /a/b. I don't want to copy it, only 1.file and 2.file so rsync on the whole directory won't work here. It can be used to copy files (which is the same as scp).
2. I have tried now splitdir and you are right, it's better than splitting by "/". Is there a subroutine that can give me all the subpaths of a path?
3. Yes it's a good subroutine but I can't use it yet because I need to parse each path - find out if it's a link. abs_path will give me the final path but I also want to have the recursive links (like I mentioned before /a/b -> /c/d -> e/f, in that case abs_path will just get /e/f and ignore /c/d).
I hope this post will clarify some opened question. If not, I will be more than glad to answer more. I'm sorry if I didn't explain the question good enough before. Thanks for the help until now!


Also, some more code that I wrote, while trying to make it work (just for reference. also really sorry it's messy and with bad variable names):
foreach my $f (@arr) { if (-l $f) { print($f, " is a link to ",readlink($f) , "\n"); my @a = split("/",$f); my $result; my $counter = 0; my $last_files_block = $st{"/"}{"files"}; while (1) { unless ($counter < scalar(@a)) { last; } my $x = $a[$counter]; if ($x eq '') { $counter += 1; next; } if ($counter + 1 == scalar(@a)) { if (-f $f) { my $found = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $found = 1; last; } } if ($found == 0) { my %vsaaa = ("type" => "link-file", "source" => re +adlink($f)); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); } my $last = $f; while (1) { my $c = readlink($last); if (-l $c) { $last = $c; if (index($c,"/") != -1) { push(@arr,$c); } else { my $found1 = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $found = 1; last; } } if ($found1 == 0) { my %vsaaa = ("type" => "file"); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); } } } else { if (index($c,"/") != -1) { push(@arr,$c); } else { my $found1 = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $found = 1; last; } } if ($found1 == 0) { my %vsaaa = ("type" => "file"); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); } } last; } } } if (-d $f) { my $found = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $found = 1; last; } } if ($found == 0) { my $n = readlink($f); if (index($n,"/") == -1) { my $dirname = dirname($f); $n = "$dirname/$n"; #TODO: what if relativ +e? } my %vsaaa = ("type" => "dir-link", "source" => + $n); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); } } last; } my $found = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $last_files_block = $v->{$x}{"files"}; $counter += 1; $found = 1; last; } } if ($found == 0) { my %vsaaa = ("type" => "dir", "files" => [] ); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); $last_files_block = $vsaaa{"files"}; $counter += 1; } } } elsif (-f $f) { print($f, " is a file\n"); my @a = split("/",$f); my $result; my $counter = 0; my $last_files_block = $st{"/"}{"files"}; while (1) { unless ($counter < scalar(@a)) { last; } my $x = $a[$counter]; if ($x eq '') { $counter += 1; next; } if ($counter + 1 == scalar(@a)) { my $found = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $found = 1; last; } } if ($found == 0) { my %vsaaa = ("type" => "file"); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); } last; } my $found = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { $last_files_block = $v->{$x}{"files"}; $counter += 1; $found = 1; last; } } if ($found == 0) { my %vsaaa = ("type" => "dir", "files" => [] ); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); $last_files_block = $vsaaa{"files"}; $counter += 1; } } } elsif (-d $f) { print($f, " is a dir\n"); my @a = split("/",$f); my $result; my $counter = 0; my $last_files_block = $st{"/"}{"files"}; while (1) { unless ($counter < scalar(@a)) { last; } my $x = $a[$counter]; if ($x eq '') { $counter += 1; next; } my $found = 0; my $found_link = 0; foreach my $v (@{$last_files_block}) { if (defined($v->{$x})) { if ($v->{$x}{"type"} eq "dir-link" || $v->{$x}{"type"} + eq "link-file") { $found_link = 1; last; } $last_files_block = $v->{$x}{"files"}; $counter += 1; $found = 1; last; } } if ($found_link == 1) { last; } if ($found == 0) { my %vsaaa = ("type" => "dir", "files" => [] ); my %st1 = ($x => \%vsaaa ); push(@{$last_files_block}, \%st1); $last_files_block = $vsaaa{"files"}; $counter += 1; } } } else { #TODO: When can it happen other than path does not exist o +r permission denied? print($f, " is a special\n"); } }

In reply to Re^2: Getting all subpaths from a path by ovedpo15
in thread Getting all subpaths from a path by ovedpo15

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.