Re^4: batch processing via single open

No, no... We are very misdirected at the moment. We have nothing to do with reading/writing from/to files. Forget about it.

I am not looking for a way to handle files. My shell script gathers the file names and passes to my Perl script. I do not want Perl to handle this instead, I am okay with it. Then my Perl file connects to "another program", executes some commands there (depending on the argument passed from shell to it) and exits. But this happens hundreds of thousands of times. Why not open the connection to "that program" only ONCE and use the same connection hundreds of thousands of times?

I certainly understand you asking for detailed information and am quite aware of posting guidelines but this is a part of an experiment for a TREC proposal. So it is impossible for me to describe such a long thesis here. If I could not clarify what I am looking for so far, then that's the limitation of my English and we have nothing to do.

One final try from me: Though I am not, let us assume the "external program" we are dealing is MySQL. That is: my Perl file opens an IPC connection to MySQL, selects a DB to work on and creates a table named ARGV[0]. That's the way it is, we cannot change it. Also assume that we want to create 100000 different tables. Now, why open a connection to MySQL for 100000 times? Let us open the connection and select the DB once at the beginning and then just create 100000 tables. It would save us a time of 99999*(time to open a new connection and select database), wouldn't it?

Still does not make sense? :(

Comment on Re^4: batch processing via single open

Replies are listed 'Best First'.
Re^5: batch processing via single open by shmem (Chancellor) on Jul 17, 2007 at 17:01 UTC
Still does not make sense? shrug Makes sense, for sure. With the MySQL abstraction at hand, go and apply that abstraction to your problem. It definitely doesn't make sense trying to help somebody who says "Please help me with doing something, but I won't tell you what it is". That said, another attempt: of course you can keep a connection to "some program" open, but only as long as the lifetime of your current perl process lasts. You can feed multiple file names to that program, but only if the calling semantics of that program support that way of feeding data, i.e. if it reads from STDIN. If your invoked program only handles arguments on its command line, there's no way, because you cannot extend the argument list after the fact. --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply]
Re^6: batch processing via single open by karden (Novice) on Jul 17, 2007 at 17:39 UTC
Ok, then I will have to open a new connection each time I run my code. That's all. I thank you so much for the trouble shmem! One correction though: I am somebody who says "Please help me with doing something; but while I really want to provide details I bet you, yourself, would not want to decode hundreds of lines of code". Thinking otherwise would be oversensitivity and mean insulting both you and me.	[reply]
Re^7: batch processing via single open by shmem (Chancellor) on Jul 17, 2007 at 17:52 UTC
Ok, then I will have to open a new connection each time I run my code. That's all. No, that's not all. Why do you think so? By which (lacking) part of my post did you come to that conclusion? Did you give up, were you answered? I thank you so much for the trouble shmem! You're welcome. But as for your next words - "Please help me with doing something; but while I really want to provide details I bet you, yourself, would not want to decode hundreds of lines of code" - these do not match the point, it's not "all or nothing". Part of my job is skimming hundreds of lines of code and get a clue of what's wrong, and I do so more often than not with code I myself perpetrated. While its not sensible to post such a bunch of lines as a node, your scratchpad will gladly store these and give interested people an opportunity to get you on the way. --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply]
Re^8: batch processing via single open by karden (Novice) on Jul 18, 2007 at 19:58 UTC