My work currently involves doing ETL (data-munging) using Perl. Because some of the data may have private information (PII), I'm doing my work on a remote desktop, which at this organization is Azure (Windows). It's not really my choice of platform -- but at least I'm able to use the excellent git-bash package to get (more or less) back to a Linux type command line.

The issue I'm having is that transferring files from my volume to the volume where my output is carried on to the next step is unreliable, and I'm wondering if anyone else has come across the same challenge. I also want to check the existing files, as I append a version number to the end of my files so that new files don't overwrite existing files.

• Copying files using File Explorer works, but sometimes a transfer will stall at 99%, at which point I may lose my connectivity to the remote desktop. That isn't a bad thing, although it irritatingly loses the network path in the destination folder, and I have to drill down through seven layers of directories to get back to where I was to check that my copy succeeded before disconnecting.
• Copying via the command line (you knew that was coming next), while logical, also appears to be hilariously slow. A file that might get copied in 15-20 seconds using Explorer takes 15-20 minutes from the command line. I have no idea why, as it should be going from one network drive to another.

The solution I've finally hit on (this is my work in progress) is to use chdir to go into the directory in question, and then glob to get a list of the files. It still takes ten seconds to get a list of files, but I guess that's the best I can do right now.

Thoughts, comments, ideas are welcome. Thanks!

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.