Re^5: unix loops in perl

I really save time with pdsh in many common task in a day to day administration.

- When you need connect to various machines (300 machines?), launch a command in that machines and get result, you won if use pdsh.

- If you need download with wget from many websites (again 300), you won using pdsh vs loop

And citing you:

"If each process takes up very little resources, then the overhead of creating them will cause it all to go slower."

Each process will be created ever you use parallel system or use an loop to create it. So no will be overhead if you have enough resources.

Of course, as I said before:

If you don't have enough CPU / Memory / network bandwith, parallel computation will not work better than loop ... even worse than it ...

Regards,

Comment on Re^5: unix loops in perl

Replies are listed 'Best First'.
Re^6: unix loops in perl by SuicideJunkie (Vicar) on Oct 27, 2011 at 14:22 UTC
You can certainly pull from multiple places, but once you've saturated your network, more connections won't help. If you do the work in a loop, you only use one process. If you create new processes and have each process do a portion of the work, then you create new processes. I don't see where that gets confusing. When I say 'each process takes up very little', I mean when it is easy for your process to just do the job itself. You've seen the classic extreme case of "lets fork off a thread to print one character" loops right? It is starting to sound like we are violently agreeing.	[reply]
Re^7: unix loops in perl by i5513 (Pilgrim) on Oct 27, 2011 at 22:32 UTC
"If you do the work in a loop, you only use one process. If you create new processes and have each process do a portion of the work, then you create new processes. I don't see where that gets confusing." Ok, in original post was : `for x in x y z ; do perl -bla 'bla bla' ; done` [download] I was talking about ;-) Some times parallelization save times, other not, it is clear. Yes, we agree, I think .. Regards,	[reply] [d/l]