[newbie question] Appending txt files with same name in different subdirectories

Kgs has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks,

I am a newb trying to figure out how I can append text files of the same name in different directories. Specifically, I have a directory with about 5000 subdirectories. Each subdirectory contains 500 text files where the name of the file is (potentially) non-unique. That is, there may be a file called name.txt in folders 000, 100, 940, and 1000.

I would like to find all instances where the file name is non-unique and APPEND them (e.g., append name.txt in folders 000, 100, 940, and 1000 above). The individual files can be deleted and the new appended file can be placed in any of the originating folders (i.e., 000, 100, 940, or 1000.)

Hopefully I have made myself clear :) I am using Strawberry Perl on Windows 10 64bit

Comment on [newbie question] Appending txt files with same name in different subdirectories

Replies are listed 'Best First'.
Re: [newbie question] Appending txt files with same name in different subdirectories by hippo (Archbishop) on Aug 17, 2016 at 10:35 UTC
One approach: Trawl through the tree with one of the File::Find::* modules. Build up a hash, keyed on the filenames with values of array refs of the directory paths. When the trawl has finished, go through the hash, ignoring any value where the array ref has a single entry. Slurp in each file in the array ref, appending to a scalar, deleting each file after you have read it, bar the last where you save the content from the scalar. Caveat 1: Depending on your hardware and the actual numbers involved you may hit memory constraints with this. I am using Strawberry Perl on Windows 10 64bit Caveat 2: Windows sometimes does some odd filesystem things so take care. (I am not an MS user so can't comment further) I am a newb Welcome! Are you new to programming or just to Perl? There is lots of documentation to help you out so do browse around and keep asking questions. Good luck and enjoy the journey.	[reply]
Re^2: [newbie question] Appending txt files with same name in different subdirectories by Kgs (Novice) on Aug 18, 2016 at 00:04 UTC
Thanks Hippo, I am new to programming, the only "coding" experience I have is with stat programs like SAS and Stata. I suppose one could say I have a slight appreciation for it. Reflecting on your advice I just came up with another method which avoids me having to learn about this stuff just yet. I just put it all the same directory (soo many files!!!) and then created a large number of folders and moved each file into a random folder (after appending).	[reply]