fasoli has asked for the wisdom of the Perl Monks concerning the following question:

Hello again, wise monks!

After reading carefully the recommendations you made in my previous question (mkdir in loop - overwrite danger? - http://www.perlmonks.org/?node_id=1150970) I decided to use make_path and a different approach. And at this point let me thank you for taking the time to reply in that question and for introducing make_path to me.

So my new plan is the following. Instead of looping through my files and matching their filenames in order to use mkdir and create the directories, I decided to use make_path in order to build the directory tree first, and then use another script to process my files and redistribute them.

I just need a bit of guidance using make_path. Let me describe what I want to do in the end so that you can tell me if it's doable by make_path. I have 150 different folders that each one of them needs to have 100 subfolders. The names of both main folders and subfolders are specific. The main folders' names are like AB0001 up to AB0030, BC0001 up to BC0050 and a couple more similar patterns of the same logic. The subfolders need to every time be S00001 up to S00100.

Is this somehow doable with make_path? I am kind of figuring out the subfolders bit as it seems easier - it's always an "S" in the beginning and I need a counter and I need to specify that the name always needs to be 6 characters long. I hope I'll be able to write that. What's confusing me is the main folders bit. Can I do the same again? Can I say that "I want 30 folders, from AB0001 up to AB0030; also 50 folders, from BC0001 up to BC0050" and so on?

Obviously I don't expect ready code, just maybe some resources and guidance until I manage to write something that runs and can be shared here and corrected. This page http://perldoc.perl.org/File/Path.html didn't help me much, it is a bit confusing, also unfortunately my two perl books don't mention make_path as they're a bit old :( That's why I'm kind of lost on how to begin.

Thank you so much all, your help is immensely appreciated.

Replies are listed 'Best First'.
Re: make_path for creating directory tree
by 1nickt (Canon) on Dec 23, 2015 at 16:05 UTC

    It sounds like your question is about program logic, not about creating folders or using make_path. Sorry if I am mistaken.

    It sounds like you need nested looping to do what you want. Also read up on range operators.

    # hash defining your folder patterns my %patterns = ( AB => 30, BC => 50, CD => 42, ); foreach my $pattern ( keys %patterns ) { foreach my $folder ( 1 .. $patterns{ $pattern } ) { # construct the folder name using the value of # $pattern, the value of $folder_name, and sprintf() # use make_path() or whatever to create the folder foreach my $subfolder ( 1 .. 100 ) { # construct the subfolder name # create the subfolder } } }

    Hope this helps!

    The way forward always starts with a minimal test.

      Yes exactly, I'm trying to figure out how to write this script, my biggest problem isn't make_path itself. I am not a programmer and I have to start every bit of programming I do by first sketching what I need to get in the end, in this case a directory tree - easy to draw, then I write in pen and paper how I think the code could look like, with one million mistakes of course, and then I try to write it. But I still lack the understanding of programming logic, you're right about that. Thanks for your answer and example, it's what I'm trying now.

Re: make_path for creating directory tree
by james28909 (Deacon) on Dec 23, 2015 at 15:06 UTC
    "Is this somehow doable with make_path?"

    You're using Perl, so ofcourse its doable!

    I would make all the base directories, push the dir name to an array, then foreach "cwd/$dir" make the sub directories how you see fit. Seems pretty much straight forward. Good Luck!

    EDIT: Maybe this will give you an idea. ;)
    use strict; use warnings; use File::Path qw(make_path); make_path("one/two/three", "four/five/six" , {verbose => 1});
    OUTPUT: C:\Perlmonks>make_paths.pl mkdir one mkdir one/two mkdir one/two/three mkdir four mkdir four/five mkdir four/five/six

      Thank you for your answer, good to hear that Perl can handle it :)

      Ok so I just tried to use make_path and I did something wrong.

      #/bin/perl/ use strict; use warnings; use File::Path qw(make_path); make_path("~/folder1/folder2/folder3/folder4");

      This creates the following directories: ~ (!!!), then folder1, then folder2, then folder3 and then folder4. If I use single quotes it just exits quietly without an error but without creating anything either. Any thoughts???

      EDIT: Sorry, we posted simultaneously! I'll study your example and retry.

        You're on linux? Im booted into windows, but anyway, are you trying to build directories in /home? If so try "/home/folder1/folder2/folder3/folder4" im pretty sure that you would need to use native 'mkdir' command to use '~/'

        EDIT: Use '~/' like this: 'mkdir -p ~/one/two/three'. I do not see any references to File::Path supporting or NOT supporting '~/'. So maybe someone else can chime in about this. :)
Re: make_path for creating directory tree
by Anonymous Monk on Dec 23, 2015 at 16:00 UTC
    Can I do the same again? Can I say that "I want 30 folders, from AB0001 up to AB0030; also 50 folders, from BC0001 up to BC0050" and so on?
    Something like this:
    use strict; use warnings; my @dirs = ( make_seq( 'AB', 6, 1, 30 ), make_seq( 'BC', 6, 1, 50 ), ); for my $dir (@dirs) { for my $subdir ( make_seq( 'S', 6, 1, 100 ) ) { print "$ENV{HOME}/$dir/$subdir\n"; } } sub make_seq { my ( $prefix, $total_length, $from, $to ) = @_; my $length = $total_length - length($prefix); my $format = "${prefix}%0${length}d"; return map sprintf( $format, $_ ), $from .. $to; }
    If that looks like what you want, replace print with make_path (and remove \n, obviously). Note that make_path doesn't understand ~ (tilde), but you can use $ENV{HOME} instead of tilde.

      Omg this is like magic :D Or like a Christmas present. It does what I want, yes. Although I don't understand some bits, so if you could be so kind and explain them you'd be helping me fill one more page in my Perl notebook! I need to understand how stuff works otherwise I'll soon be asking the same things again and annoy you all.

      So:

      1. my $length = $total_length - length($prefix);

      my $format = "${prefix}%0${length}d";

      ^

      What is happening here? So in the first line, you subtract the length of the prefix from the total length, which is 6. Ok, I got that. But then in the second line, I don't get it. Why "${prefix}" and not "{$prefix}"? Oh I think I just got the next bit: "%0${length}d" -> that says we need "length"-many zeros, whatever the value of $length is from above.

      2. return map sprintf( $format, $_ ), $from .. $to;

      ^

      I'm really lost with that one. If the for loop that goes through @dirs already does the job, shouldn't return map be before the for loop? Also why is it return map and not map?

      3. Actually in general what confused me is that I thought Perl reads the script line by line from top to bottom. Now in this example we first use make_path and then set the subroutine. Shouldn't the subroutine be on top of everything else?

      I know you probably get this all the time but I do apologise for the silly questions - I have already started reading the stuff on subroutines, map, make_path and so on, but a bit of discussion helps greatly.

      Thank you for the help, this page is amazing for solving Perl problems and answering questions.

        I need to understand how stuff works otherwise I'll soon be asking the same things again and annoy you all.

        AnonyMonk has given a pretty good explanation above, but you need to understand where to look stuff up. That way, you'll be able to answer your own questions when the time comes that everyone else is too busy or too annoyed to answer them for you.

        ... I thought Perl reads the script line by line from top to bottom. Now in this example we first use make_path and then set the subroutine. Shouldn't the subroutine be on top of everything else?

        In Perl, a subroutine can, in most cases, be defined anywhere in the program and at any time (compile- or run-time) just as long as it is defined when it is actually called at runtime (see perlintro and perlsub). The only examples I can think of that require prior definition or declaration of a function involve prototypes — but don't involve yourself with prototypes unless you have to; see Far More than Everything You've Ever Wanted to Know about Prototypes in Perl -- by Tom Christiansen.


        Give a man a fish:  <%-{-{-{-<

        Although I don't understand some bits, so if you could be so kind and explain them you'd be helping me fill one more page in my Perl notebook!
        Sure. Feel free to ask any questions.
        my $length = $total_length - length($prefix); my $format = "${prefix}%0${length}d";
        $format just makes a format string for sprintf. It seems you're not familiar with that function. It is modeled after the famous printf function in the C programming language. I recommend you to just search the internet for something like "printf tutorial". There must be lots of those, and it's a very useful function in general.

        The goal of the line my $format = "${prefix}%0${length}d"; is to produce a string like "AB%04d", which is a format string for sprintf.

        Why "${prefix}" and not "{$prefix}"?
        ${length} is actually a more useful example here. You know that you can interpolate variables in strings like so:"$variable". Suppose that I used $length instead of ${length}: "${prefix}%0$lengthd". Then Perl would've tried to find variable $lengthd (notice the d at the end), and there's no such variable. You can use curly braces to disambiguate: "${length}d". Now it's clear (to Perl) that $ goes with length, while d is just a character and is not a part of the variable's name.

        And why ${prefix} instead of just $prefix? Just for symmetry with ${length}.

        Another way to write it:
        my $format = $prefix . '%0' . $length . 'd';
        Actually in general what confused me is that I thought Perl reads the script line by line from top to bottom. Now in this example we first use make_path and then set the subroutine. Shouldn't the subroutine be on top of everything else?
        Actually, no. Variables work like that, but not subroutines. I always put all subroutines at the bottom. That's different from some other programming languages and UNIX shells - in those, functions must also be put on top before you can actually use them - but not in Perl.
        return map sprintf( $format, $_ ), $from .. $to;
        This is a pretty dense line. map is another famous function, from functional programming languages. You should read it right-to-left (like in Arabic):
        $from .. $to
        That creates an anonymous list of numbers, starting with $from and ending with $to. For example, if $from is 1 and $to is 5, it will create (1,  2, 3, 4, 5). It's anonymous because it doesn't have a name and exists only temporarily in Perl's virtual machine.
        map sprintf( $format, $_ ), ANONYMOUS_LIST
        map acts like a loop. It loops over anonymous list and invokes sprintf for each element of that list. map puts the result into another anonymous list. Like so:
        for (ANONYMOUS_LIST) { STRING = sprint( $format, $_ ); put STRING into ANOTHER_ANONYMOUS_LIST; }
        and then return returns ANOTHER_ANONYMOUS_LIST from the make_seq function.
      Hi all! I have been trying to edit the above example in order to create a directory tree that goes: structure/molecule/step. As in, /struct1/AB0001/S00001/ and /struct2/BC0001/S00001/. So, struct1 corresponds only to molecule AB, and struct2 corresponds only to molecule BC. However, both AB and BC files should have the same "steps" subdirectories in them, each time from S00001 to whatever step we are calculating at the time. With that in mind, I figured to first create the "parent" directories struct1 and struct2 and then, in another loop, to put in the relevant subdirectories. This however gives me the error "Use of uninitialized value $structs in concatenation (.) or string".
      #/bin/perl/ use strict; use warnings; use File::Path qw(make_path); my @structs; my $structs; my @molecs; my $molecs; my @steps; my $steps; # Parent structure directory. @structs = ( dir_tree( 'struct', 7, 1, 1), dir_tree( 'struct', 7, 2, 2 ), # dir_tree( 'struct', 7, 3, 3 ), # dir_tree( 'struct', 7, 4, 4 ), # dir_tree( 'struct', 7, 5, 5 ), # dir_tree( 'struct', 7, 6, 6 ), ); # Subfolder molecule directory. @molecs = ( dir_tree( 'AB', 6, 1, 1 ), dir_tree( 'BC', 6, 1, 1 ), # dir_tree( 'XZ', 6, 1, 4 ), # dir_tree( 'XY', 6, 1, 4 ), # dir_tree( 'QU', 6, 1, 4 ), # dir_tree( 'QE', 6, 1, 4 ), ); # Subfolder steps directory. @steps = ( dir_tree( 'S', 6, 1, 2 ), ); for $structs (@structs) { make_path "/media/RAIDstorage/home/results/$structs"; } for $molecs (@molecs) { for $steps (@steps) { make_path "/media/RAIDstorage/home/results/$structs/$molecs/$s +teps"; } } # Define subroutine here. sub dir_tree { my ( $name, $total_length, $from, $to ) = @_; my $length = $total_length - length($name); my $format = "${name}%0${length}d"; return map sprintf( $format, $_ ), $from .. $to; }
      If I use an integrated loop, like so
      for $structs (@structs) { for $molecs (@molecs) { for $steps (@steps) { make_path "/media/RAIDstorage/home/results/$structs/$molecs/$s +teps"; } } }
      I get no errors but the wrong directory tree. That way, all $molecs are added in both $structs, when I want AB to go only in $struct1 and BC to go only in $struct2. I also tried using two different subroutines, like sub1 and sub2, but it also gave me the same error, "uninitialized value". Is the answer the two different subroutines, only I did it wrong? Any hints as to what I'm messing up? Thank you all!
        Oops sorry, it looks like I forgot to log in when posting the above.
      Reviving an old thread, but I wanted to ask if it is possible to have a different increment instead of 1, for $from .. $to in the above script? An increment of 5 or 10 or something? Like when in a for loop one could use for ($i=1; $i=100; $i+5) or something like that?