Run one perl script from another?

punch_card_don has asked for the wisdom of the Perl Monks concerning the following question:

Messianic Monks,

I have your basic database management interface written in Perl and using Template Toolkit. Create a record, update a record, list records by criteria, etc etc.

Now, I want to upload a bunch of data from a text file to the database. The text file's structure does not mirror the database's, so there's going to have be a parsing routine written anyway - so , I thought, wouldn't it be cool to format the newly parsed data into a string encoded as if the html 'create record' form was submitted, then call the existing 'create record' script and pass the string to it.

But - (a) how to do that; (b)what will become of the confirmation template the creation script outputs; (c) what have I not even thought of?

Thanks

Forget that fear of gravity,
Get a little savagery in your life.

Comment on Run one perl script from another?

Replies are listed 'Best First'.
Re: Run one perl script from another? by guinex (Sexton) on Jun 20, 2007 at 19:30 UTC
I suppose cool is in the eye of the beholder :) I would just refactor your form processing script so that it contains a function that takes the parsed data and inserts it into the database. Both the form processing code and the text parsing code could then call this same function. -guinex	[reply]
Re: Run one perl script from another? by talexb (Chancellor) on Jun 20, 2007 at 21:12 UTC
A saner approach ;) would be to use something like LWP::UserAgent to automate the procedure of filling in the form and submitting it. Then, you could use something like HTML::Parser to create your own parser, so that you could sift through the resulting 'page' and make sure nothing bad happened. I have a script that does batch document uploads to my web application using this technique, and it seems to work quite well. Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply]
Re: Run one perl script from another? by Grundle (Scribe) on Jun 20, 2007 at 22:16 UTC
It seems to me that he is saying that there is no HTML involved in this process, but that he is basically spoofing a forms submission so that he can call a create record script that was created for a previous web-based database application. Doing it this way is only going to complicate things for you. Would it be cool? It might be a nice hack, but what happens when your load increases on this type of parsing. Are you going to abuse all your text this way and force it through a non-essential forms submission, or you could bite the bullet once and just write a passthrough script to submit directly to the database. Plus you have to double parse so that you can get the data and then format it into your encoded submission. You have the right idea about reusable code, but make sure it applies to what you are doing. I would probably just parse the data according to the requirements and then use DBI for my database operations.	[reply]
Re: Run one perl script from another? by graff (Chancellor) on Jun 21, 2007 at 03:59 UTC
If you have shell access to the database server, your best bet is to write a really simple perl script that just converts the original text file into another plain-text format that can be loaded directly into the table via the database server's native "import" or "load" tool. If your text file is just a few (hundred) records, you might not notice the speed difference, but if it runs into the many thousands of records, the 10-to-1 or worse wall clock ratio (DBI inserts vs. bulk insert using the right tool) will be significant. This is equally true for mysql, oracle, postgres, etc. Regardless of the number of records involved, you'll spend a lot less time programming, and it will be a lot easier to get right. (If you don't have shell access, but are using mysql, it is still likely to be easier/better to upload a properly formatted version of the text file and use "LOAD DATA INFILE ...", rather than a do bunch of DBI inserts.)	[reply]